question

Andreasss-5315 avatar image
0 Votes"
Andreasss-5315 asked Andreasss-5315 commented

How to round down to nearest stepping/precision interval problem

Hello!

I am trying to understand how I should do to round down numbers with precision intervals.

I will explain the scenario: I have a value and a _precision.

1. First example:
I have the value 0.0006 and a _precision of 0.0001. I now want to round downto the nearest stepping value of 0.0001 from 0.0006 which is 0.0006. My formula does it correct here!

2. Second example:
value: 0.0001234567
_precision: 0.00013
Stepping down 0.00013 from 0.0001234567 will be -0,0000065433. But we only want values which is >= 0. So 0 should be the correct answer in this scenario. But my formula gives the wrong number: 0.00013.

3. Third example:
value: 0.0002512345
_precision: 0.00013
The nearest interval _precision of 0.00013 below 0.0002512345 is: 0.00013 which should be the
correct answer. But my formula gives the wrong number: 0.00026.


I wonder how I should write the result formula to take care of those 3 scenarios?

             double value = 0.0006;
             double _precision = 0.0001;
             double result1 = value - value % _precision + ((value % _precision < _precision / 2) ? 0.0 : _precision);
             ///
             double value2 = 0.0001234567;
             double _precision2 = 0.00013;
             double result2 = value2 - value2 % _precision2 + ((value2 % _precision2 < _precision2 / 2) ? 0.0 : _precision2); 
             ///
             double value3 = 0.0002512345;
             double _precision3 = 0.00013;
             double result3 = value3 - value3 % _precision3 + ((value3 % _precision3 < _precision3 / 2) ? 0.0 : _precision3);
    
             //result1: 0.0006   (This is correct!)
             //result2: 0.00013   (This should be rounded down to 0)
             //result3: 0.00026   (This should be rounded down to 0.00013)
             MessageBox.Show(result1 + "___" + result2 + "___" + result3);
dotnet-csharp
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

Viorel-1 avatar image
0 Votes"
Viorel-1 answered Andreasss-5315 commented

Is it possible to to use decimal instead of double?

 decimal value = 0.0006m;
 decimal _precision = 0.0001m;
 decimal result1 = Math.Floor( value / _precision ) * _precision;
 ///
 decimal value2 = 0.0001234567m;
 decimal _precision2 = 0.00013m;
 decimal result2 = Math.Floor( value2 / _precision2 ) * _precision2;
 ///
 decimal value3 = 0.0002512345m;
 decimal _precision3 = 0.00013m;
 decimal result3 = Math.Floor( value3 / _precision3 ) * _precision2;
    
 Console.WriteLine( result1 + "___" + result2 + "___" + result3 );

· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

That did work, thank you.

I think I could use decimals, I need to look at the logic around the function but it should work.
I tried to change decimal to double in your example but if I do that it will not be correct.

I wonder why decimal actually produce the correct result and not double?

0 Votes 0 ·

Double cannot represent the numbers exactly. For example:

 double value1 = 0.00013;
 Console.WriteLine( value1.ToString( "G20" ) ); // Displays "0.00012999999999999999"

Decimal works better in this case. This type is also suitable for financial calculations.

There are differences between the range of double and decimal numbers, which are described in documentation.

0 Votes 0 ·

Thank you Viorel-1

That helped alot!

I will use decimals in this scenario.

Thank you for your help!

0 Votes 0 ·