Skip to content

Conversation

@tompng
Copy link
Member

@tompng tompng commented Sep 7, 2025

Almost all methods in BigDecimal/BigMath that takes precision calculates in relative precision.
Only BigMath.sin(except x around 0) and BigMath.cos(always) was calculating in fixed-point precision.
There is no document that says sin and cos is fixed-point precision. It can be considered bug (or wrong document).

BigMath.sin(BigDecimal('31.4159265358979323846264338327951111111111'), 30)
#=> 0.822691394060062514...e-31 (before, precision: 18)
#=> 0.822691394060062489417902505540769218359371379100137196517465784...e-31 (after)
#=> 0.82269139406006248941790250554076921835937137910013719651746578736517692e-31 (actual value)

Other

  • Implementing BigMath.tan needs relative precision version of sin and cos.
  • BigMath.tan(x, prec) will calculate in relative precision in all x around int * π / 2, so if sin and cos is in fixed-point precision, it's not consistent.

@tompng tompng merged commit 514d36c into ruby:master Sep 11, 2025
81 checks passed
@tompng tompng deleted the relative_sin_cos branch September 11, 2025 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant