Reason for the mismatch between sys.dm_os_memory_nodes.virtual_address_space_committed_kb and aggregated sys.dm_os_memory_clerks.virtual_memory_committed_kb by memory_node_id

A PFE colleague presented the following scenario and asked for the justification of what seemed to be, a priori, something inexplicable. The story was like this:

When I check the memory allocation using memory nodes and memory clerks DMVs the total amount of committed VM reported is a bit different. I can see similar difference around reserved VAS.

select memory_node_id as node,

virtual_address_space_committed_kb/(1024) as VirtualMemoryCommittedMB,

single_pages_kb/(1024) as SinglePagesMB,

multi_pages_kb/(1024) as MultiPagesMB,

locked_page_allocations_kb/(1024) as AWEPagesMB,

shared_memory_committed_kb/(1024) as SharedMemoryMB,

virtual_address_space_reserved_kb/(1024) as VAS_reserved_mb

from sys.dm_os_memory_nodes

where memory_node_id != 64

SELECT

memory_node_id as node,

SUM(virtual_memory_committed_kb)/1024 AS VirtualMemoryCommittedMB,

SUM(single_pages_kb)/1024 AS SinglePagesMB,

SUM(multi_pages_kb)/1024 AS MultiPagesMB,

SUM(awe_allocated_kb)/1024 AS AWEPagesMB,

SUM(shared_memory_committed_kb)/1024 AS SharedMemoryMB,

SUM(virtual_memory_reserved_kb)/1024 AS VAS_reserved_mb

FROM sys.dm_os_memory_clerks

WHERE memory_node_id != 64

GROUP BY memory_node_id

The results are as follows:

Does anyone know the reason? Are there any allocation not done through the clerks?

The real reason (based on what I read in the code, not based in my guessing or anything I’ve found from any esoteric source available in the Wild Wide Web ) why those values do not match is the following:

sys.dm_os_memory_nodes exposes aggregated commitment at the node level, regardless of the clerk that may have interfaced (and accounted for) the allocation.

Whereas sys.dm_os_memory_clerks exposes commitment at the clerk level BUT (and here comes the important part that justifies the difference), the enumerator it uses to scan the list of clerks only exposes those of type 1 only (TYPE_MEMORYCLERK) and not those of types 2, 3, and 4 (TYPE_CACHESTORE, TYPE_USERSTORE, and TYPE_OBJECTSTORE).

Therefore, whatever charge any clerk of those three last types has put on any memory node is visible through the sys.dm_os_memory_nodes, but not through the sys.dm_os_memory_clerks.

What I refer to type 1 (TYPE_MEMORYCLERK) in this post is what Slava referred to as “generic” memory clerks in the Memory Clerks section of his SQLOS's memory manager and SQL Server's Buffer Pool post.