Skip to content

Conversation

@tompng
Copy link
Member

@tompng tompng commented Jul 9, 2025

Fixes #202
(Some test already passes now, some test that still fail seems not a bug or a bug also in DECDIG=uint32_t)

if (VpVtoD(&d, &e, v.real) != 1)
    return rb_float_new(d);
// Range of d is: 10**-BASE_FIG <= d < 1.0

// Overflow if d*10**e > 10**DBL_MAX_10_EXP = 1e+308
if (e > (SIGNED_VALUE)(DBL_MAX_10_EXP+BASE_FIG))
    goto overflow;
// Overflow if d*10**e < 10**(DBL_MAX_10_EXP - DBL_DIG) = 1e-323
if (e < (SIGNED_VALUE)(DBL_MIN_10_EXP-DBL_DIG))
    goto underflow;  

When DECDIG is uint32_t, e is a multiple of 9. (example: -315, -324)
In this case, result of wrong calculation (e < (SIGNED_VALUE)(DBL_MIN_10_EXP-9)) and correct calculation (e < (SIGNED_VALUE)(DBL_MIN_10_EXP-15)) was the same just by chance.

@tompng tompng marked this pull request as draft July 9, 2025 09:49
@tompng tompng force-pushed the fix_to_float_underflow_uint16 branch from e802064 to 214e0bc Compare July 9, 2025 12:51
@tompng tompng marked this pull request as ready for review July 9, 2025 13:07
@tompng tompng merged commit d246504 into ruby:master Jul 10, 2025
155 checks passed
@tompng tompng deleted the fix_to_float_underflow_uint16 branch July 10, 2025 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bugs when DECDIG == uint16_t

1 participant