Closed Bug 1813 Opened 26 years ago Closed 20 years ago

Javascript has Pentium-style precision errors

Categories

(Core :: JavaScript Engine, defect, P2)

defect

Tracking

()

VERIFIED DUPLICATE of bug 5856

People

(Reporter: akkzilla, Assigned: mike+mozilla)

References

()

Details

(This is probably a dup of a known bug ...) 25.4 * 3 comes out 76.1999999... instead of 76.2. See URL for demo. Happens on all versions of our browser up to and including 5.0, and on all platforms we've tried so far.
Status: NEW → RESOLVED
Closed: 26 years ago
Resolution: --- → INVALID
Clayton, our IEEE wiz, reports: Sigh. Perhaps we really do need decimal or rational arithmetic. Surely this is the same old story. 4/10 == 2/5 isn't exactly representable in binary. Conversion to binary and subsequent multiplication by 3 (probably) creates a binary bit pattern whose nearest decimal representation (to 16 digits) is 1.199999999..... Even the I18N package might fix this since rounding to fewer digits would give the user what she expects. It's not a universal panacea, but delivering fewer digits would stop 99.999....% of these complaints. = C = The basic problem is that you're seeing machine arithmetic. The problem isn't Pentium-style, it's universal to binary representations of numbers. (Which javascript currently uses.) We're dreaming about a decimal-arithmetic package, but it's not currently scheduled.
Javacsript component begin retired. Moving this bug to Javascript Engine.
Marking Verified/Invalid
Status: RESOLVED → VERIFIED
Verified Invalid
Status: VERIFIED → REOPENED
Resolution: INVALID → ---
*** This bug has been marked as a duplicate of 5856 ***
Status: REOPENED → RESOLVED
Closed: 26 years ago20 years ago
Resolution: --- → DUPLICATE
Verifying
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.