HIVE-29217: Add configuration to choose materialization strategy for CTEs#6092
HIVE-29217: Add configuration to choose materialization strategy for CTEs#6092zabetak merged 6 commits intoapache:masterfrom
Conversation
zabetak
force-pushed
the
cte-cbo-ast-procs
branch
from
3284f0b to
eb80d40
Compare
asf-ci-hive
added
tests pending
tests unstable
and removed
tests unstable
tests pending
labels
zabetak
force-pushed
the
cte-cbo-ast-procs
branch
from
eb80d40 to
8fc8884
Compare
asf-ci-hive
added
tests pending
tests unstable
and removed
tests unstable
tests pending
labels
zabetak
force-pushed
the
cte-cbo-ast-procs
branch
from
8fc8884 to
a7da724
Compare
|
zabetak
force-pushed
the
cte-cbo-ast-procs
branch
from
a7da724 to
bda9d4e
Compare
asf-ci-hive
added
tests pending
tests unstable
and removed
tests passed
tests pending
labels
zabetak
force-pushed
the
cte-cbo-ast-procs
branch
from
bda9d4e to
927feb1
Compare
asf-ci-hive
added
tests pending
tests unstable
and removed
tests unstable
tests pending
labels
zabetak
force-pushed
the
cte-cbo-ast-procs
branch
from
927feb1 to
820c6af
Compare
deniskuzZ
left a comment
deniskuzZ
left a comment
There was a problem hiding this comment.
HiveConf default AST, but we test with CBO, is that intentional?
zabetak
force-pushed
the
cte-cbo-ast-procs
branch
from
820c6af to
9709e8b
Compare
|
@deniskuzZ Many thanks for the review. I believe I addressed all your comments so please take another look once you get the chance.
Regarding the test coverage for The AST is the default value and is tested as part of the main plan regression suite (TezTPCDS30TBCliConfig) as well as everywhere else where the config is not overridden. The CBO is an experimental feature, and the main reason that we introduced a separate CTE suite (TPCDSCteCliConfig). |
zabetak
force-pushed
the
cte-cbo-ast-procs
branch
from
9709e8b to
3b9a25e
Compare
| HiveTableScan(table=[[cte, cte_suggestion_2]], table:alias=[cte_suggestion_2]) | ||
| HiveProject(d_date_sk=[$0]) | ||
| HiveFilter(condition=[BETWEEN(false, CAST($2):TIMESTAMP(9), 1998-08-04 00:00:00:TIMESTAMP(9), 1998-09-03 00:00:00:TIMESTAMP(9))]) | ||
| HiveFilter(condition=[AND(<=(1998-08-04 00:00:00, CAST($2):TIMESTAMP(9)), <=(CAST($2):TIMESTAMP(9), 1998-09-03 00:00:00))]) |
There was a problem hiding this comment.
is this also related to https://issues.apache.org/jira/browse/HIVE-29469?
1. Avoid the need to maintain lots of duplicated content inside XML config files. 2. Enhance the focus of the test by clearly separating the important properties
The test case in this file explicitly targets CBO suggestions.
zabetak
force-pushed
the
cte-cbo-ast-procs
branch
from
3b9a25e to
20fb495
Compare
|
|
Thanks for the review @deniskuzZ ! Much appreciated. |
3 participants
Why are the changes needed?
Avoid interference between AST and CBO based materialization of common table expressions (CTEs). More details in HIVE-29217.
Does this PR introduce any user-facing change?
Yes but highly unlikely. Users may see plan/perf changes if they set/rely on the experimental property
hive.optimize.cte.suggester.classand they expect both AST and CBO materialization to trigger sequentially one after the other.How was this patch tested?
Existing plus new tests.