-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix imcompatible between presto and velox when hash double/float -0.0 #9511
Fix imcompatible between presto and velox when hash double/float -0.0 #9511
Conversation
✅ Deploy Preview for meta-velox ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
@bikramSingh91 @rschlussel Similar to NaN, how does Presto treat 0.0 and -0.0? It seems 0.0 = -0.0 returns true, but during join/groupby they are treated differently? Should we add this to our plan of NaN unification? |
Agree that we need 1st to check what we want it to be in Presto. |
@spershin thats probably true based on this comment in folly |
do you have an example query to reproduce this? Earlier I had the same suspicion about group by and tried to test it out, but didn't see the behavior you are describing. |
> select 0.0 = -0.0;
_col0
-------
true
> WITH t as (SELECT * FROM unnest(array[0.0, -0.0], array[1, 2]) as _t(k, v)) select * from t join t as t2 on t.k = t2.k;
k | v | k | v
------+---+------+---
0.0 | 1 | 0.0 | 1
-0.0 | 2 | -0.0 | 2 |
With this change, do we need to normalize -0.0 and 0.0 in #9272 for Spark? @zhli1142015 |
Yes, we need to also normalize -0.0 and 0.0. If this is correct behavior for Presto, then Presto and spark expect very different behaviors when handling Nans and -0.0 / 0.0. Should we continue the #9272, @mbasmanova? thanks. |
This is the behavior of mysql, like presto, it also won't aggregate -0 and 0.
|
This pull request has been automatically marked as stale because it has not had recent activity. If you'd still like this PR merged, please comment on the PR, make sure you've addressed reviewer comments, and rebase on the latest main. Thank you for your contributions! |
For double/float -0.0 and 0.0, folly::hasher will return the same value.
but in presto/mysql, the return hash values are not equal. We should firstly
cast double/float to longBits, then call folly::hasher to get hash value.