Skip to content

feat[expr]: N-ary CASE WHEN expression#6786

Merged
robert3005 merged 7 commits intovortex-data:developfrom
spiceai:lukim/case-when
Mar 4, 2026
Merged

feat[expr]: N-ary CASE WHEN expression#6786
robert3005 merged 7 commits intovortex-data:developfrom
spiceai:lukim/case-when

Conversation

@lukekim
Copy link
Contributor

@lukekim lukekim commented Mar 4, 2026

Summary

Introduces CASE WHEN expression support in Vortex as a scalar function, implementing a true n-ary CASE WHEN cond1 THEN val1 WHEN cond2 THEN val2 ... ELSE default END expression.

Changes included

This pull request introduces support for SQL-style CASE WHEN expressions in Vortex, including both the expression logic and integration with DataFusion. The changes add new expression constructors, conversion logic, and pushdown support for CASE WHEN, as well as comprehensive benchmarks and tests to ensure correctness and performance.

CASE WHEN expression support:

  • Added new constructors case_when, case_when_no_else, and nested_case_when to build CASE WHEN expressions in exprs.rs, leveraging the new CaseWhen scalar function and its options. [1] [2]
  • Introduced CaseWhenOpts protobuf message to encode CASE WHEN options for serialization.
  • Registered the new case_when scalar function module.

DataFusion integration:

  • Implemented conversion from DataFusion's CaseExpr to Vortex's CASE WHEN expressions, supporting the "searched CASE" form and mapping WHEN/THEN/ELSE clauses. [1] [2]
  • Enhanced pushdown logic to support CASE WHEN expressions, including recursive checks for child expressions and ELSE clauses. [1] [2]

Benchmarking and testing:

  • Added comprehensive benchmarks for various CASE WHEN scenarios in case_when_bench.rs and registered them in Cargo.toml. [1] [2]
  • Added an equivalence test to ensure that DataFusion CASE WHEN results match Vortex's results for the same input data.

Other improvements:

  • Refined scalar function pushdown checks and utility logic for expression convertibility.
  • Minor code quality improvements and error handling in the expression module.

Testing

  • Added a comprehensive test to verify equivalence between DataFusion and Vortex results for a CASE WHEN expression applied to an Arrow RecordBatch.

Benchmarking

  • Introduced a new benchmark for various CASE WHEN scenarios, including simple, nested, all-true, and all-false cases, in benches/expr/case_when_bench.rs, and registered it in Cargo.toml. [1] [2]
Benchmark 1K rows 10K rows 100K rows
case_when_simple 5.4 µs 8.4 µs 29.5 µs
case_when_without_else 5.5 µs 8.7 µs 29.4 µs
case_when_all_true 4.2 µs 6.3 µs 20.7 µs
case_when_all_false 4.3 µs 6.4 µs 20.1 µs
case_when_nary_3_conditions 15.1 µs 25.8 µs 87.3 µs
case_when_nary_equality_lookup (5) 25.2 µs 39.7 µs 125.4 µs
case_when_nary_10_conditions 45.5 µs 73.1 µs 257.5 µs

lukekim added 7 commits March 4, 2026 08:44
…13)

* feat: implement binary CASE WHEN expression with support for nested conditions
Signed-off-by: Luke Kim <80174+lukekim@users.noreply.github.com>
…ub.com>

I, Luke Kim <80174+lukekim@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 9538417
I, Luke Kim <80174+lukekim@users.noreply.github.com>, hereby add my Signed-off-by to this commit: c648553
I, Luke Kim <80174+lukekim@users.noreply.github.com>, hereby add my Signed-off-by to this commit: c897c84
I, Luke Kim <80174+lukekim@users.noreply.github.com>, hereby add my Signed-off-by to this commit: af601cd
I, Luke Kim <80174+lukekim@users.noreply.github.com>, hereby add my Signed-off-by to this commit: 929d7b3

Signed-off-by: Luke Kim <80174+lukekim@users.noreply.github.com>
@robert3005 robert3005 added the changelog/feature A new feature label Mar 4, 2026
Copy link
Contributor

@robert3005 robert3005 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In order to retain Luke's contribution attribution we will merge this and then fix one small bug that we had on the original pr

@robert3005 robert3005 merged commit df23c37 into vortex-data:develop Mar 4, 2026
59 of 60 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants