/    Sign up×
Community /Pin to ProfileBookmark

Inexplicable Arithmetic

Copy linkTweet thisAlerts:
Nov 13.2006

I have absolutely no idea what’s going on here.

Here’s a function…

[code=html]function addit(n)
{
pr = “price” + n;
qt = “qty” + n;
tt = “tot” + n;
p = document.getElementById(pr).value;
q = document.getElementById(qt).value;
t = p * q;
document.getElementById(tt).value = t;
}[/code]

It collects the price and quantity values, multiplies them, and writes them back into a total. Simple enough? I wish!

The price entered was 226.20.
The quantity entered was 3.
The result generated was 678.5999999999999.

Anyone got an explaination?

to post a comment
JavaScript

3 Comments(s) ↴

Copy linkTweet thisAlerts:
@CharlesNov 13.2006 — Anyone got an explaination?[/QUOTE]Well, the long and the short of it is that computers are not calculators and they are bad at math. It has to do with converting between base 2 and base 10 and using only a limited degree of accuracy for memory reasons.
Copy linkTweet thisAlerts:
@mrhooNov 13.2006 — The explanation is in the way floating point arithmetic works.

To return the results you want:

t= (Math.round(p*q*100)/100).toFixed(2)
Copy linkTweet thisAlerts:
@chestertbauthorNov 13.2006 — Thanks guys. That's more or less what I thought. MathRound it is.

I used to tell my clients, back in the deep dark days of DOS, that computers hand a fundamental problem... they don't work. I told them that as long as they started from that basic pre-conception, everything they got from there was a bonus. They thought I was joking.

Cheers

CTB
×

Success!

Help @chestertb spread the word by sharing this article on Twitter...

Tweet This
Sign in
Forgot password?
Sign in with TwitchSign in with GithubCreate Account
about: ({
version: 0.1.9 — BETA 6.16,
whats_new: community page,
up_next: more Davinci•003 tasks,
coming_soon: events calendar,
social: @webDeveloperHQ
});

legal: ({
terms: of use,
privacy: policy
});
changelog: (
version: 0.1.9,
notes: added community page

version: 0.1.8,
notes: added Davinci•003

version: 0.1.7,
notes: upvote answers to bounties

version: 0.1.6,
notes: article editor refresh
)...
recent_tips: (
tipper: @nearjob,
tipped: article
amount: 1000 SATS,

tipper: @meenaratha,
tipped: article
amount: 1000 SATS,

tipper: @meenaratha,
tipped: article
amount: 1000 SATS,
)...