Closed Bug 594836 Opened 14 years ago Closed 13 years ago

mac intel hardware produces different results for math.tan() with 16 decimal places

Categories

(Tamarin Graveyard :: Virtual Machine, defect)

x86
macOS
defect
Not set
normal

Tracking

(Not tracked)

VERIFIED WONTFIX

People

(Reporter: brbaker, Unassigned)

References

Details

Running Math.tan(0.6108652375000001) on mac intel hardware (tested with Core2 duo and CoreDuo) produces a value of: 0.7002075371694642 While other platforms (linux32/64, windows32/64. linux ARM, mac PPC) generate: 0.7002075371694643 This may just be a fact of life, but issue was discovered while trying to have the jsbench/RayTracer.as performance test self validate (bug #526234)
Flags: flashplayer-triage+
Flags: flashplayer-qrb?
Flags: flashplayer-injection-
you could write a simple C program to test the underlying tan() function. if that also varies the same way, mystery solved. otherwise it could be something subtle having to do with 80/64 bit precision at different points. I would expect variation between x86-32 (which uses mixed 64/80 precision) and x86-64, which uses 64-bit exclusively. but i would not expect mac-specific variation.
Not only could Math.tan compute values with higher / lower precision, but the higher / lower precision could in principle affect the number->string code in Tamarin, and / or the various bugs in the latter code could manifest differently. If you want to eliminate the output problem (until I've fixed those bugs) you can write the Number value to a ByteArray and thus compare the resulting bits directly.
(In reply to comment #1) > you could write a simple C program to test the underlying tan() function. if > that also varies the same way, mystery solved. printf("%4.16f\n", tan (0.6108652375000001)); linux: 0.7002075371694643 mac intel: 0.7002075371694642 mac ppc: 0.7002075371694643 Looks like this is a fact of life?
May want to test in the Flash Player, to see if the Flash Player somehow controls the floating point width (and to see if we should, too).
(In reply to comment #4) > May want to test in the Flash Player, to see if the Flash Player somehow > controls the floating point width (and to see if we should, too). Player produces the same results as tamarin shell, different values: mac intel: 0.7002075371694642 windows : 0.7002075371694643 Checked with both argo (10.1) and coral (10.0) players, and the results did not change.
QRB: Should this just be resolved as WONTFIX?
FOL, marking as Resolved WONTFIX.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → WONTFIX
Flags: flashplayer-qrb? → flashplayer-qrb+
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.