Floating Point Calculation Error

In a blog post I read somewhere that if you give alert(9.2*100); in JavaScript then the alert box will show 919.999999999999 instead of 920.

Many people wonder, “Why is it so?”, “Is it a bug?”

Answer to question “Is it a Bug?” is Yes and No.

Yes, because you are getting wrong output and a general definition of bug is “software bug is an error, flaw, mistake, failure, or fault in a computer program or system that produces an incorrect or unexpected result, or causes it to behave in unintended ways.”

No, because this is the way floating point calculations have been designed and this is what has been standardized by IEEE in IEEE 754 standard. Similar behavior you will see in 8.2*100, .1+.2 etc…

To Understand “Why is it so?”

The behavior we are seeing is because underneath calculations are done in Binary not in Decimal and the languages which don’t handle rounding off of numbers show this behavior.

Let’s have a look at this calculation

9.2 * 100 = 1001.? * 1100100

Converting .2 into Binary

.2 * 2 = 0.4 – 0

.4 *2 = 0.8 – 0

.8*2 = 1.6 – 1

.6*2 = 1.2 – 1

 

So number is – 00110011……(an infinite chain)

Calculating back to Decimal: to show that we are not taking huge precision and it will impact final result

2^(-1) *0+ 2^(-2)*0+2^(-3) *1+ 2^(-4)*1 +2^(-5) *0+ 2^(-6)*0+2^(-7) *1+ 2^(-8)*1

= (.5)*0+(.25)*0+(.125)*1+(.0625)*1+(.03125)*0+(.015625)*0+(.0078125)*1+(.00390625)*1

= 0+0+.125+.0625+0+0+.0078125+.00390625

= 0.19921875

 

Now 9.2 is 1001.00110011….

Or Take it as 100100110011 after incorporating shift (2^8)

Now 100100110011 * 1100100 = 111001011111101100 (Done in Programmer Calculator J)

Incorporating Shift

1110010111.11101100

1110010111
= 919

11101100
= .5+.25+.125+0+.03125+.015625+0+0…………………

= 0.921875

So 1110010111.11101100 is 919.921875

If we would have taken a longer chain of.2 i.e. we would have taken 00110011001100110011001100110011001100110011001100110 it would have given a more precise result of 919.99999999999999

To read more about floating point errors you can check these links.

http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

http://docs.python.org/2/tutorial/floatingpoint.html

Leave a Reply

Your email address will not be published. Required fields are marked *