Problem for Convert a Single to Decimal or string

Question

Hello,

I'm having a problem converting a single to a decimal or a string.

In the example, I'm trying to convert a single passed as a parameter into a character string (for example).

' if Valeur = 11764.7666     
Private Function NbDecimalCalcul(ByVal Valeur As Single) As Int16         
	Dim Dec As Decimal = Valeur             ' return 11764.77D    !          
	Dim Str As String = Valeur.ToString     ' return "11764.77"   !      
End Function

The conversion doesn't work correctly. The simple is rounded to 2 digits.

See screenshots

Thx.

Steeve

Visual Basic 2010 - Express

Answer 1

floats are not precise. also they are stored as base 2, and typically are converted to base 10 to display. this is an issue because some base 10 factions do not convert to base 2, they are irrational. similar 1/3 is .33333... in decimal. so you do not get the same number when you convert a base10 literal fraction to Float (base 2), and then back to base10 to display. also depending on the digits Float has ~7 to 9 decimal digits of precision.

https://en.wikipedia.org/wiki/Single-precision_floating-point_format

in .net, a conversion of Float to Decimal only has 7 digits of precision by design (see docs). you can pick up a few more digits of precision by converting to Double first, though you will may get a different base 10 number when its converted to base 10 (as in your case).

Dim Valeur As Single = 11764.7666
Dim Dbl as Double = Valeur '11764.7666015625
Dim Dec As Decimal = Dbl   '11764.7666015625

in short if you need accurate decimal fractions, never use floating point numbers.

Answer 2

Hi @ Steeve,

You can use "R" format string to preserve the precision of the Single type.

Dim Str As String = Valeur.ToString("R") ' return "11764.7666"

Best Regards.

Jiachen Li

---

If the answer is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 3

Also try this:

Dim Str2 As String = Valeur.ToString("G9") ' return "11764.7666"

• Wayne

Answer 4

Hi @WayneAKing and @Jiachen Li-MSFT

Thank you for your feedback.

So why doesn't it always work!

Example:
The S3 case, when assigned, trims a digit

I must be tired!

Best Regards.

Steeve

Answer 5

>So why doesn't it always work!

>Example:

>The S3 case, when assigned, trims a digit

Probably because of precision differences and limitations between Single and Double types.

Note that it is the Single variable S3 which is showing a value that is different than what your code specifies.

The numeric literal 11764.6667 is a Double type by default. So here you are seeing a down-cast conversion from a Double to a Single. This results in a probable loss of precision. To see how this affects the value stored, try adding a type suffix letter to the literal to declare it as a Single type literal rather than the default Double type. You can do that by appending either an F or a ! to the numeric literal.

Dim S3 As Single = 11764.6667F

Dim S3 As Single = 11764.6667!

When you do that watch what happens to that literal as soon as you move away from it in the IDE editor. You may see it suddenly change in your source code to

Dim S3 As Single = 11764.667F

or

Dim S3 As Single = 11764.667!

• Wayne