Closed
Bug 480475
Opened 16 years ago
Closed 16 years ago
JavaScript Math is producing incorrect results with floating point differences
Categories
(Core :: JavaScript Engine, defect)
Tracking
()
People
(Reporter: cmtalbert, Unassigned)
Details
So, I hope you can tell me this bug is already filed or expected in some odd way. It seems really broken to me, and I couldn't find anything in bugzilla that is still open and would explain it.
Here's the issue (performed with build pulled & built today, 2/26 at 15:00 or so PST):
First, the original problem:
js> Math.abs(3.73205-3.73206);
0.000010000000000065512
Should be: 0.00001
Then simplifying:
js> Math.abs(1.001-1.002);
0.001000000000000112
js> Math.abs(1.01-1.02);
0.010000000000000009
js> Math.abs(1.1-1.2);
0.09999999999999987
Without abs():
js> 1.1-1.2
-0.09999999999999987
js> 1.2-1.1
0.09999999999999987
js>
Expected: the math should be correct. There really shouldn't need to be any rounding errors with something as simple as 1.2-1.1, but I don't know how you've implemented this.
Marking blocking ? as the ability to do simple math seems pretty critical.
Flags: in-testsuite?
Flags: blocking1.9.2?
Dammit, forgot to mention: tried this on Windows, Mac OS X and Linux with mozcentral, and got similar results across the board. The results from the JS shell are on an intel macBook running OS X 10.5.
Another friend falls to ieee-binary!
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → DUPLICATE
Updated•16 years ago
|
Flags: in-testsuite?
Flags: blocking1.9.2?
You need to log in
before you can comment on or make changes to this bug.
Description
•