question

John-6058 avatar image
0 Votes"
John-6058 asked MinxinYu-MSFT commented

How to make Visual Studio UI show the precise underlying float point numbers?

Hi all,

 int main() {
     float a = 123.45;
     return 0;
 }

By default, the UI value will show an approximated number.

What's the option to show the precise underlying number (123.4499969482421875 rather than 123.44997)?

202368-image.png


c++
image.png (11.9 KiB)
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

John-6058 avatar image John-6058 DavidLowndes-6766 ·

Sorry, maybe I should use the word "precise", I have changed the context.

0 Votes 0 ·

Given that a float only has around 7 significant digits, what you get is as precise as it can be.
Why do you think the number you quoted is what it should be?

0 Votes 0 ·
Show more comments

1 Answer

MinxinYu-MSFT avatar image
0 Votes"
MinxinYu-MSFT answered John-6058 commented

Hi, @John-6058

You need to use double a = 123.45f; instead of float.
13 decimal places for double are reliable.
the difference:
123.44999694824219 //13 . 14 is not unreliable
123.4499969482421875
202545-image.png
Best regards,

Minxin Yu


If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.




image.png (5.1 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks, but it's not what I want.

0 Votes 0 ·