Decimal Arithmetic

Feb 25, 2010 at 3:42 PM

Running:

 

object result = ( new JintEngine( ) ).Run( "var a= 0.1+0.2; return a;" );

returns: 0.30000000000000004

Any ideas????

 

Feb 25, 2010 at 4:55 PM

This has to do with the precision of computers.  In the same way that 1/3 has no precise representation in decimal, 1/10 and 2/10 have no precise representation in binary.  You're actually performing 1 / 1010 + 1 / 10100, and both of those are repeating numbers.  Since computers can't store an infinite number of bits of data (sadly!), you're getting the rounded approximation.

1/3 ~= .333333, 2/3 ~= .666666, 1 ~= .999999, but 1 != .999999.

I hope that helps.  I've run into this exact thing before.

Feb 26, 2010 at 3:21 PM
Edited Feb 26, 2010 at 3:23 PM

I know what you mean, and this is particularly relevant for JavaScript (there is a whole debate in ECMA about this point (http://www.infoq.com/presentations/The-State-and-Future-of-JavaScript)). But the hosting language has the option to handle it better.

For example, in C#:

 

            decimal a = 0.1M;
            decimal b = 0.2M;
            decimal c = a + b;

            double d = 0.1;
            double e1 = 0.2;
            double f = d + e1;

            float g = 0.1F;
            float h = 0.2F;
            float i = g + h;

If you put a break point, and watch the values in the locals window, then c and i give the expected result (0.3), but f (a double) has the same problem as JavaScript.

I guess my suggestion is for jint to use the decimal data type to represent JavaScript numbers.