VBA CDec Function Conversion Question

James:

Can someone provide some insight to the first step of my function

First, the exponential operator (^) always returns type Double.  To confirm, note that the following displays Double:

MsgBox TypeName(CDec(2) ^ CDec(56))

Type Double can represent 2^56 accurately with no loss of precision (72057594037927936).

(Type Double can represent any power of 2 up to 2^1023 accurately with no loss of precision.)

And CDec(255) * CDec("72057594037927936") is exactly 18374686479671623680, the accurate result with no loss of precision.

However, CDec(255) * CDec(2^56) is 18374686479671614500 (!).

I believe the reason is:  CDec(double) first formats the type Double expression, which is limited to 15 significant digits.

Note that Format(2^56,"0") is 72057594037927900.  And CDec(255) * CDec("72057594037927900") is indeed 18374686479671614500.

The largest power of 2 that can be formatted accurately with 15 significant digits (or less) is 2^49 (562949953421312).

So you could write CDec(255) * 2^49 * 2^7, but not CDec(255) * 2^50 * 2^6.

Or you could simply write:

Sub doit()

Const conRELMacAppEnu As Byte = 255

Dim varRELAudMsgSvc As Variant, varShift56 As Variant

varShift56 = CDec("72057594037927936")

varRELAudMsgSvc = conRELMacAppEnu * varShift56

MsgBox varRELAudMsgSvc

End Sub

Hi.  In general, the power function "^" is not decimal compliant.  2^56 is too high.

If you want to use "^", the last example is something I would do. (but not recommended)

Sub Demo()

Dim K As Variant

Dim d1 As Variant

Const conRELMacAppEnu As Byte = 255

Dim varRELAudMsgSvc As Variant

K = 2 ^ 28

d1 = CDec(1)

varRELAudMsgSvc = CDec(conRELMacAppEnu)

Debug.Print "Wrong"

Debug.Print varRELAudMsgSvc * 2 ^ 56

Debug.Print "Correct"

Debug.Print varRELAudMsgSvc * K * K

Debug.Print "Also correct, but not recommended"

Debug.Print (2 ^ 8 - d1) * 2 ^ 28 * 2 ^ 28

End Sub

Returns:

Wrong

 18374686479671614500 

Correct

 18374686479671623680 

Also correct

 18374686479671623680