double roundoff error in calculations

Question

double roundoff error in calculations

Lawrence254 0

All of these following code statements should yield the same value, but the second to fourth statement have a second to last digit rounding error in c#. The first statement is correct and fifth to last are correct. Do you know this is happening?

Console.WriteLine(360.0 / 365.2421875);

Console.WriteLine(360.0 + 360.0 / 365.2421875 - 360.0);

Console.WriteLine(360.0 * (365.2421875 + 1.0) / 365.2421875 % 360.0);

Console.WriteLine((360.0 / (365.2421875 / (365.2421875 + 1.0))) % 360.0);

Console.WriteLine((decimal)(360.0m / 365.2421875m));

Console.WriteLine((decimal)(360.0m + 360.0m / 365.2421875m - 360.0m));

Console.WriteLine((decimal)(360.0m * (365.2421875m + 1.0m) / 365.2421875m % 360.0m));

Console.WriteLine((decimal)((360.0m/ (365.2421875m / (365.2421875m + 1.0m))) % 360.0m));

Console.WriteLine(360.0m / 365.2421875m);

Console.WriteLine(360.0m + 360.0m / 365.2421875m - 360.0m);

Console.WriteLine(360.0m * (365.2421875m + 1.0m) / 365.2421875m % 360.0m);

Console.WriteLine((360.0m / (365.2421875m / (365.2421875m + 1.0m))) % 360.0m);

Lawrence254 0 Reputation points

2024-05-11T20:36:12.88+00:00

See the second to last digit in the second to the forth line. It only happens when doing addition or subtraction with DOUBLES. The first line is correct and the fifth to the end I was using DECIMAL in those code lines to show that the first line is correct and the second to the forth have an error in the second to the last digit. Here is my run of the code:

0.985647365831747

0.985647365831767

0.985647365831767

0.985647365831767

0.9856473658317469145045025775

0.98564736583174691450450258

0.98564736583174691450450258

0.98564736583174691450450258

0.9856473658317469145045025775

0.98564736583174691450450258

0.98564736583174691450450258

0.98564736583174691450450258
David Lowndes 2,640 Reputation points MVP

2024-05-12T11:34:02.9333333+00:00
If we simplify the example:

Console.WriteLine(360.0 / 365.2421875); // 0.985647365831747` Console.WriteLine(360.0 / 365.2421875 + 360.0 - 360.0); // 0.9856473658317668` Console.WriteLine(360.0 / 365.2421875 + (360.0 -360.0)); // 0.985647365831747`

It's interesting to note that the bracketed expression gives a different result to the one without brackets, and is the same output as the simple division only one.

The disassembly shows the generated code as:

Console.WriteLine(360.0 / 365.2421875); // 0.985647365831747 vmovsd xmm0,qword ptr [DoubleTest.Program.Main(System.String[])+058h (07FF8C0677118h)] call qword ptr [CLRStub[MethodDescPrestub]@00007FF8C08CF360 (07FF8C08CF360h)] nop Console.WriteLine(360.0 / 365.2421875 + 360.0 - 360.0); // 0.9856473658317668 vmovsd xmm0,qword ptr [DoubleTest.Program.Main(System.String[])+060h (07FF8C0677120h)] call qword ptr [CLRStub[MethodDescPrestub]@00007FF8C08CF360 (07FF8C08CF360h)] nop Console.WriteLine(360.0 / 365.2421875 + (360.0 -360.0)); // 0.985647365831747 vmovsd xmm0,qword ptr [DoubleTest.Program.Main(System.String[])+058h (07FF8C0677118h)] call qword ptr [CLRStub[MethodDescPrestub]@00007FF8C08CF360 (07FF8C08CF360h)] nop

Note that the first and last use the same memory location whereas the second is different. Since the calculation isn't being done at run-time, the value's been generated by the compiler.

It appears to me that the compiler has for some unknown reason generated a different result - possibly bracketing the expression causes it to know the outcome is 0 and so doesn't actually do 360.0 - 360.0 which is presumably where the loss of precision comes about.

I think it'd be worthwhile reporting it using VS's Report a Problem facility to see what MS say about it; the results aren't what I'd expect. Be sure to supply a simple example and clear steps so as to get past the triage stage.

If you do report it, please post a link to your report back here so we can follow it.

2 answers

Your answer

Lawrence254 0 Reputation points

2024-05-11T20:36:12.88+00:00

See the second to last digit in the second to the forth line. It only happens when doing addition or subtraction with DOUBLES. The first line is correct and the fifth to the end I was using DECIMAL in those code lines to show that the first line is correct and the second to the forth have an error in the second to the last digit. Here is my run of the code:

0.985647365831747

0.985647365831767

0.985647365831767

0.985647365831767

0.9856473658317469145045025775

0.98564736583174691450450258

0.98564736583174691450450258

0.98564736583174691450450258

0.9856473658317469145045025775

0.98564736583174691450450258

0.98564736583174691450450258

0.98564736583174691450450258

Answer 1

With the Decimal type, all arithmetic is integer with implied decimal point. So it can accurately do decimal arithmetic. decimal has 28-29 significant digits depending on the number.

With double, all arithmetic is done in base 2 using floating point math (which can lose precision) and is converted to base 10 on display. This conversion is only approximate because not all base 2 fractions convert to decimal fractions. Double has 15-17 significant digits depending on the number.

double because it’s a digit plus exponent can represent very large and very small numbers.

Answer 2

Bruce (SqlWork.com) 78,006 Volunteer Moderator

your results are expected.

Not all decimal (base 10) fractions can be accurately converted to double (base 2). Just like 1/3 can not be converted to a decimal fraction, many decimal fractions can not be accurately converted to base 2.

There is often approximation when a decimal numbers (with fractions) are converted to double (or float) and again when a double is converted to decimal for display.

Lawrence254 0 Reputation points

2024-05-13T22:08:38.8166667+00:00

I'm not talking about DECIMALs, only DOUBLEs. The decimals are only there to show that the first line has the correct answer. The next 3 lines have rounding errors from the addition and subtraction in those lines. FORGET about the next 8 lines, they are only to prove the ERROR.
Anonymous

2024-05-14T09:27:31.9133333+00:00

Hi @Lawrence254 , Welcome to Microsoft Q&A,

What do you want to express? You can add your own predicates to increase floating point precision.

Share via

double roundoff error in calculations

2 answers

Your answer