Closed Bug 480475 Opened 16 years ago Closed 16 years ago

JavaScript Math is producing incorrect results with floating point differences

Categories

(Core :: JavaScript Engine, defect)

x86
All
defect
Not set
normal

Tracking

()

VERIFIED DUPLICATE of bug 5856

People

(Reporter: cmtalbert, Unassigned)

Details

So, I hope you can tell me this bug is already filed or expected in some odd way. It seems really broken to me, and I couldn't find anything in bugzilla that is still open and would explain it. Here's the issue (performed with build pulled & built today, 2/26 at 15:00 or so PST): First, the original problem: js> Math.abs(3.73205-3.73206); 0.000010000000000065512 Should be: 0.00001 Then simplifying: js> Math.abs(1.001-1.002); 0.001000000000000112 js> Math.abs(1.01-1.02); 0.010000000000000009 js> Math.abs(1.1-1.2); 0.09999999999999987 Without abs(): js> 1.1-1.2 -0.09999999999999987 js> 1.2-1.1 0.09999999999999987 js> Expected: the math should be correct. There really shouldn't need to be any rounding errors with something as simple as 1.2-1.1, but I don't know how you've implemented this. Marking blocking ? as the ability to do simple math seems pretty critical.
Flags: in-testsuite?
Flags: blocking1.9.2?
Dammit, forgot to mention: tried this on Windows, Mac OS X and Linux with mozcentral, and got similar results across the board. The results from the JS shell are on an intel macBook running OS X 10.5.
Another friend falls to ieee-binary!
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → DUPLICATE
Verified my friend!
Status: RESOLVED → VERIFIED
Flags: in-testsuite?
Flags: blocking1.9.2?
You need to log in before you can comment on or make changes to this bug.