[SPARK-33903][SQL] Add CREATE VIEW AS SELECT support for V2 ViewCatalog#55042
Open
viirya wants to merge 3 commits intoapache:masterfrom
Open
[SPARK-33903][SQL] Add CREATE VIEW AS SELECT support for V2 ViewCatalog#55042viirya wants to merge 3 commits intoapache:masterfrom
viirya wants to merge 3 commits intoapache:masterfrom
Conversation
Route CREATE VIEW and DROP VIEW through the V2 execution path when the target catalog implements ViewCatalog. **Changes:** - `ResolveSessionCatalog`: allow `CreateView` and `DropView` to fall through (instead of throwing) when the target catalog implements `ViewCatalog`; non-`ViewCatalog` V2 catalogs still throw the capability error. - `DataSourceV2Strategy`: add routing cases for `CreateView` and `DropView` targeting a `ViewCatalog`. For CREATE VIEW, the aliased schema and current catalog/namespace are pre-computed from the session and passed to the exec. - `CreateViewExec`: new `LeafV2CommandExec` that validates the view body (no temp-object references, no auto-generated aliases) and calls `ViewCatalog.createView` / `replaceView` with a `ViewInfo` built from the analyzed query schema, SQL text, and user properties. - `DropViewExec`: new `LeafV2CommandExec` that calls `ViewCatalog.dropView`. - `InMemoryViewCatalog`: in-memory `ViewCatalog` + `SupportsNamespaces` implementation for testing. - `CreateViewV2Suite`: tests for basic create, OR REPLACE, IF NOT EXISTS, duplicate-view error, non-ViewCatalog error, and comment/ property propagation. Known limitation (TODO): temp-view reference validation is skipped in the V2 exec path because the optimizer inlines view bodies before `CreateViewExec.run()`. A follow-up should prevent optimizer traversal into `CreateView.query` for V2 targets (similar to `AnalysisOnlyCommand` in the V1 path). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…operties - Expose ViewHelper.viewSchemaModeToProps so CreateViewExec can call it; WITH SCHEMA BINDING / EVOLUTION is now stored as view.schemaMode in ViewInfo. - Capture active SQL configs (e.g. ANSI mode, session timezone) at creation time via ViewHelper.sqlConfigsToProps, mirroring V1 generateViewProperties. - Add TODO comment documenting the referred-temp-object-names gap. - Add tests for schema mode and SQL config capture in CreateViewV2Suite. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…V2View Introduce CreateV2View, a new logical plan node that extends AnalysisOnlyCommand, to replace the CreateView fall-through path when targeting a ViewCatalog. - AnalysisOnlyCommand hides the view query from the optimizer once analyzed, so View(isTempView=true) nodes remain intact and can be detected by verifyTemporaryObjectsNotExists. - markAsAnalyzed captures referredTempFunctionNames from AnalysisContext, which is now threaded through to CreateViewExec and passed to verifyTemporaryObjectsNotExists. - ResolveSessionCatalog produces CreateV2View (instead of falling through) when CreateView targets a ViewCatalog. - DataSourceV2Strategy matches on CreateV2View instead of CreateView. - Un-ignore the temp-view rejection test in CreateViewV2Suite. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Route CREATE VIEW and DROP VIEW through the V2 execution path when the target catalog implements ViewCatalog.
Changes:
ResolveSessionCatalog: allowCreateViewandDropViewto fall through (instead of throwing) when the target catalog implementsViewCatalog; non-ViewCatalogV2 catalogs still throw the capability error.DataSourceV2Strategy: add routing cases forCreateViewandDropViewtargeting aViewCatalog. For CREATE VIEW, the aliased schema and current catalog/namespace are pre-computed from the session and passed to the exec.CreateV2View: a new logical plan node that extendsAnalysisOnlyCommand, to replace theCreateViewfall-through path when targeting aViewCatalog.CreateViewExec: newLeafV2CommandExecthat validates the view body (no temp-object references, no auto-generated aliases) and callsViewCatalog.createView/replaceViewwith aViewInfobuilt from the analyzed query schema, SQL text, and user properties.DropViewExec: newLeafV2CommandExecthat callsViewCatalog.dropView.InMemoryViewCatalog: in-memoryViewCatalog+SupportsNamespacesimplementation for testing.CreateViewV2Suite: tests for basic create, OR REPLACE, IF NOT EXISTS, duplicate-view error, non-ViewCatalog error, and comment/ property propagation.Why are the changes needed?
CREATE VIEW AS SELECT only worked with the V1/session catalog. When targeting a V2 catalog that implements ViewCatalog, Spark threw an error despite the ViewCatalog API (createView, replaceView, ViewInfo) already being fully in place. The missing piece was the execution plumbing from SQL parsing through to that API.
Does this PR introduce any user-facing change?
Yes.
CREATE VIEW AS SELECTsupports V2 ViewCatalog after this PR.How was this patch tested?
Unit test
Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Sonnet 4.6