-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Support different scales for decimal binary math functions #19874
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Introduce calculate_binary_math_decimal and calculate_binary_math_numeric functions with a proper handling of arguments of different scales and type casting. They supersede calculate_binary_math and calculate_binary_math_decimal due to having a slightly different functior signature with automatic passing of effective precision and scale (even if rescaled).
|
I'm having trouble understanding the rationale here; Also for |
For most of these functions, agree, only one argument is decimal, so we execute the decimal/non-decimal case with code for adjusting both scales untouched. Overall, I can see the following benefits:
So, symmetric functions like gcd/lcm (WIP), mod, div, etc benefit most from this PR. From the first glance, a For
Yes, I discovered it recently. I am wondering whether it could also be simplified using this PR, since it handles different input and output precisions and scales. |
|
I agree log can be greatly simplified, and it's something I'd love to get around to eventually (same for power), however I'm not sure if this is the way to do it; the code introduced here at a glance is not trivial and it doesn't solve any use case at hand (in that nothing is being simplified by this PR). Perhaps it would be easier to see the use case if one of the functions which could benefit from this are refactored to use it in a way that has clear benefits over the existing |
Which issue does this PR close?
Rationale for this change
A helper
calculate_binary_math andUDFs relying on it could behave strangely if the scales of inputs and outputs are different. Original logic didn't fully handle it.So let's introduce
calculate_binary_math_decimalandcalculate_binary_math_numeric functionswith a proper handling of arguments of different scales and type casting for input and output arguments.They supersede
calculate_binary_mathandcalculate_binary_math_decimalbecause they have a slightly different functor signature that automatically passes the effective precision and scale (even if rescaled). The rest is compatible.What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?
Older functions could be deprecated. Since they are a part of the public interface of datafusion-functions, I just placed a comment without a full-fledged
deprecatemacro. Up to discussion whether it should be used