JoeYoest wrote: I am curious whether they are actually performing the quadrature with good enough error bounds or if they're just interpolating between values in a pre-calculated table.
Unless Microsoft has documented the internal algorithm -- which is unlikely -- no one in this peer-to-peer forum can answer the question dispositively. Only an Excel engineer would know. AFAIK, they never participate in this forum. And if they did, they
are probably not permitted to divulge such proprietary information.
JoeYoest wrote: I'm not sure how many digits of the Excel result I can 'trust'. Excel will display out to the 9th decimal place, but I don't know if that's accurate.
In a nutshell, we cannot say, a priori, how many decimal digits of the result of Excel NORM.DIST can be "trusted".
There is a difference between "accuracy" and "precision".
We cannot know how "accurate" the Excel NORM.DIST results are without comparing them to a third-party implementation that can be considered a benchmark.
For example, Excel PI displays the value 3.14159265358979. The exact decimal representation of the binary value is 3.14159265358979,3115997963468544185161590576171875. But one website [3] says that pi to the same number of decimal places
is actually 3.14159265358979,3238462643383279502884197169399375.
(I use period for the decimal point and comma to demarcate the first 15 significant digits.)
So we might say that Excel PI is accurate to 15 decimal places (16 significant decimal digits). But that is an accident of implementation [4]. And note that Excel formats only up to 15 significant digits (rounded), anyway -- 14 decimal places for this
example.
On the other hand, all Excel numeric values are "precise" to 53 binary digits -- that is, the sum of 53 consecutive powers of 2 times an exponential factor, which can be represented by 1 to 716 [1] significant decimal digits.
But the "accuracy" of such precision is an illusion. To demonstrate, consider the decimal fraction 0.23.
By definition, it is "accurate" to 2 decimal places.
But the exact decimal representation of the "precise" binary value is 0.230000000000000,0099920072216264088638126850128173828125.
Moreover, note that =(100.23 - 100 = 0.23) returns FALSE(!) [2].
The reason is: the binary representation of the decimal fraction part of 100.23 is 0.230000000000003,9790393202565610408782958984375.
The difference arises because some of the 53 consecutive powers of 2 must be used to represent the integer part (100), leaving the sum of fewer powers of 2 to approximate the decimal fraction part.
[1] The exact decimal representation of the smallest (normalized) binary value, 2^-1022, is 1023 decimal digits after the decimal point; that is, 307 zeros followed by 716 decimal digits.
[2] We can minimize such anomalies by explicitly rounding to the accuracy that we expect. For example,
=(ROUND(100.23 - 100**, 2**) = 0.23).
[3] I did not vet the website that provides the first 100,000 digits of pi.
[4] The exact decimal representation of the binary value for 3.14159265358979 is 3.14159265358979,0007373494518105871975421905517578125.
But the exact decimal representation of the binary value for 3.1415926535897932 is
3.14159265358979,3115997963468544185161590576171875.
So that is as close to the 17-decimal-digit representation of pi that we can get with 64-bit binary floating-point.
We can enter 3.1415926535897932 into Excel with the formula =3.14159265358979+32E-16. Or we can use VBA to return CDbl("3.1415926535897932"). VBA processes all digits, not just the first 15 significant digits, as Excel does.