Skip to content

Upgrade to sqlparser 0.61.0#20177

Draft
alamb wants to merge 3 commits intoapache:mainfrom
alamb:alamb/test_sqlparser_update
Draft

Upgrade to sqlparser 0.61.0#20177
alamb wants to merge 3 commits intoapache:mainfrom
alamb:alamb/test_sqlparser_update

Conversation

@alamb
Copy link
Contributor

@alamb alamb commented Feb 5, 2026

DRAFT until SQL parser is released

Which issue does this PR close?

Rationale for this change

Keep up to date with dependencies

I think @Samyak2 specifically would like access to the : field syntax

What changes are included in this PR?

  1. Update to 0.61.0
  2. Update APIs

Are these changes tested?

Yes by existing tests

Are there any user-facing changes?

New dependency

@github-actions github-actions bot added the sql SQL Planner label Feb 5, 2026
@alamb
Copy link
Contributor Author

alamb commented Feb 5, 2026

There are a few sqllogictest errors like this

External error: 35 errors in file /Users/andrewlamb/Software/datafusion2/datafusion/sqllogictest/test_files/map.slt

1. statement failed: DataFusion error: SQL error: ParserError("Expected: ), found: { at Line: 3, Column: 8")
[SQL] CREATE TABLE map_array_table_1
AS VALUES
  (MAP {1: [1, NULL, 3], 2: [4, NULL, 6], 3: [7, 8, 9]}, 1, 1.0, '1'),
  (MAP {4: [1, NULL, 3], 5: [4, NULL, 6], 6: [7, 8, 9]}, 5, 5.0, '5'),
  (MAP {7: [1, NULL, 3], 8: [9, NULL, 6], 9: [7, 8, 9]}, 4, 4.0, '4')
;
at /Users/andrewlamb/Software/datafusion2/datafusion/sqllogictest/test_files/map.slt:18

@alamb alamb force-pushed the alamb/test_sqlparser_update branch from c95affc to 93fa56e Compare February 6, 2026 13:11
@github-actions github-actions bot added optimizer Optimizer rules core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) common Related to common crate labels Feb 6, 2026
@alamb
Copy link
Contributor Author

alamb commented Feb 6, 2026

I had Codex make up a DataFusion specific dialect to get the tests passing. I now need to clean it up, Occam's razor style

@Samyak2
Copy link
Contributor

Samyak2 commented Feb 6, 2026

Should we perhaps consider the errors as bugs in GenericDialect instead? There's some SQL that was previously parsed successfully by GenericDialect but fails now -- sounds like something to fix

@Samyak2
Copy link
Contributor

Samyak2 commented Feb 6, 2026

I have an issue and a PR for this upstream: apache/datafusion-sqlparser-rs#2204 and apache/datafusion-sqlparser-rs#2205

Seems to fix all the 35 errors here

@alamb
Copy link
Contributor Author

alamb commented Feb 6, 2026

Should we perhaps consider the errors as bugs in GenericDialect instead? There's some SQL that was previously parsed successfully by GenericDialect but fails now -- sounds like something to fix

Yes, likely -- this is great news -- thank you @Samyak2

@alamb alamb force-pushed the alamb/test_sqlparser_update branch from 93fa56e to 8bda485 Compare February 6, 2026 21:00
@github-actions github-actions bot removed optimizer Optimizer rules core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) common Related to common crate labels Feb 6, 2026
@alamb alamb force-pushed the alamb/test_sqlparser_update branch from 8bda485 to 315e7c0 Compare February 6, 2026 21:13
@alamb alamb force-pushed the alamb/test_sqlparser_update branch from 315e7c0 to 0461c5a Compare February 6, 2026 21:13
@github-actions github-actions bot added the sqllogictest SQL Logic Tests (.slt) label Feb 6, 2026
}) => {
if temporary {
return not_impl_err!("Temporary tables not supported")?;
return not_impl_err!("Temporary tables not supported");
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a drive by cleanup -- no need to return and ? -- just return is fine

@alamb
Copy link
Contributor Author

alamb commented Feb 6, 2026

Should we perhaps consider the errors as bugs in GenericDialect instead? There's some SQL that was previously parsed successfully by GenericDialect but fails now -- sounds like something to fix

Yes, likely -- this is great news -- thank you @Samyak2

Thanks again @Samyak2 -- I updated this branch to include the fix (and the 0.61.0 Release candidate) and all the tests pass now.

Thanks 🙏 !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sql SQL Planner sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants