Skip to content

compare ints against doubles without rounding#1377

Open
pranit0808 wants to merge 1 commit into
cel-expr:masterfrom
pranit0808:compare-double-int-exact
Open

compare ints against doubles without rounding#1377
pranit0808 wants to merge 1 commit into
cel-expr:masterfrom
pranit0808:compare-double-int-exact

Conversation

@pranit0808

Copy link
Copy Markdown
  1. compareDoubleInt and compareDoubleUint converted the integer operand to a double before comparing, so any int or uint above 2^53 was rounded onto a neighbouring value first.
  2. Equal is defined as Compare() == 0 for all three numeric types, so int(9007199254740993) == 9007199254740992.0 evaluated to true and MaxInt64 < 9223372036854775808.0 evaluated to false.
  3. That also disagreed with map and list indexing, which already resolve cross-type numeric keys through the lossless helpers in overflow.go: with those same two values x in [d] returned true while {d: 1}[x] reported no such key.

Split the double instead of converting the integer: range-check it, compare the integral part directly against the integer operand, and let the remaining fraction break the tie. NaN never reaches either helper, every caller screens it first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant