Skip to content

Conversation

@tompng
Copy link
Member

@tompng tompng commented Jul 29, 2025

Follow up of #347
BigMath.exp(rational, prec) was converting Rational to BigDecimal with BigDecimal(rational, 0)
Change it to BigDecimal(rational, prec).

BigDecimal(8).power(1/3r, 40)
#=> 0.1999999999999999999999999999999986137056e1 (master)
#=> 0.2e1 (v3.2.2 and this branch)

Existing problem

Using prec to coerce is not sufficient in some case.

# Needs to coerce 30000001/3r with prec=1009 to calculate correct value
BigMath.exp(30000001/3r, 1000)

@tompng tompng force-pushed the bigmath_coerce_prec_fix branch 2 times, most recently from d785685 to 55a5bc5 Compare July 29, 2025 18:45
@tompng tompng force-pushed the bigmath_coerce_prec_fix branch from 55a5bc5 to 774d09b Compare July 29, 2025 18:58
@tompng tompng merged commit ed9e91a into ruby:master Jul 29, 2025
79 checks passed
@tompng tompng deleted the bigmath_coerce_prec_fix branch July 29, 2025 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant