Closed Bug 154103 Opened 22 years ago Closed 22 years ago

arithmetic error in JavaScript Engine

Categories

(Core :: JavaScript Engine, defect)

x86
Linux
defect
Not set
normal

Tracking

()

VERIFIED INVALID

People

(Reporter: fab, Assigned: rogerl)

References

()

Details

From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1a) Gecko/20020610 BuildID: 20020610 multiplication of 0.4 with 3 results in 1.2000000000000002 haven't found other numbers leading to false results yet. Reproducible: Always Steps to Reproduce: try to multiply 0.4 with 0.3 in javascript Actual Results: the operation resultet in: 1.2000000000000002 Expected Results: if i am correct it should be exactly 1.2 :-) i tried to reproduce with mozilla 1.1a and the debian mozilla 1.0 package. both show the same error. created a simple testcase (URL)
Status: UNCONFIRMED → NEW
Ever confirmed: true
I'm not sure how 1.2 can possibly be the correct result since there's at least one value between 1.2 and 0.4 * 3.0.
IE5.5 gives the answer as 1.2000000000000001. Probably caused because the numbers are represented as floating-point (read:not exact) numbers.
<input type="button" value="0.8 * 3 = " onclick="alert( 0.8 * 3 );"> gives 2.40000000000000000004 on NT 4 (2002062108). A bit too much ;-)
Have to mark this one invalid; this is the way floating-point arithmetic works in any computer language (e.g c/c++) that adheres to the IEEE 754 standard. The point of all this is the following: computers do arithmetic in base 2, not base 10. Certain numbers which have finite decimal expansions in base 10 have infinite decimal expansions in base 2. That, plus the fact that the computer must round off infinite representations to finite ones, produces the results above. Compare these bugs, where the same issue was raised: bug 20140 "Rounding Error?" bug 42134 "JavaScript should be more lenient comparing (==) floats" bug 129300 "Error in JavaScript internal routine: parseFloat()" Please see these for more explanations and references -
Status: NEW → RESOLVED
Closed: 22 years ago
Resolution: --- → INVALID
Summary: arithmetic error in javascript engine → arithmetic error in JavaScript Engine
Marking Verified - Fabian: thank you for this report, and please continue to use Bugzilla. This is an issue that catches many people by surprise (including me!)
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.