Numeric Value Storage Precision In Memory

Question

Hello,

I’m trying to clarify my understanding of how M stores number values (i.e. how they’re represented in memory), in part so that I can clearly explain this to others.

The language specification says, “A number is represented with at least the precision of a Double (but may retain more precision).”

What are the rules controlling when M will decide to retain more precision? (The specification allows the option for M to retain more precision, but leaves ambiguous if/when this will indeed occur.)

Is the logic used here something simple like:

• If the incoming numeric value (e.g. numeric literal in user's M code or number value received from data source) can be exactly represented as a double then store as a double.
• Else if input value can be exactly represented as a decimal, then store as a decimal.
• Else fall back to storing as a double, accepting the loss of precision that will occur.

Clarity here would be much appreciated so that I can explain this with preciseness instead of with a disconcerting statement like “M can possibly lose precision on numbers that don’t fit precisely in a double but I can’t tell you predictably when this will happen, just watch out for it.”

A related aside: When M talks about decimal precision, would that be effectively equivalent to .Net’s decimal ((-296 to 296) / 10(0 to 28))?

Thank you,
Ben

(For clarity, this question is about the precision M uses when it stores numeric values in memory, not about the precision it uses when performing mathematical operations. Per the language spec, M defaults the latter to double precision, unless explicitly instructed otherwise.)

Answer 1

Hi @Ben ,

Ben Gribaudo has blog about it here: https://bengribaudo.com/blog/2018/07/31/4497/power-query-m-primer-part7-types-numbers , specifically in the section: How accurate is accurate enough? (Precision)

Best
— Miguel Caballero