Volume Shadow Copy Causing Paged Pool Memory Consumption Until Server Becomes Unresponsive

Background

Our goal was to have a month of hourly business-hours VSCs so we bumped up our Volume Shadow Copy retention. Now, two of our file servers are running high Committed Memory and eventually become unresponsive if no action is taken. I'm hoping someone has an idea of why we are seeing this issue.

• The maximum number of Volume Shadow Copies (VSC) was recently increased from 64 to 384
• VSCs are taken hourly from 9a-9p.
• Taking new VSCs consumes paged pool memory that is not released until the servers are rebooted.
• Limiting the number of VSCs stops paged pool growth as older VSCs are removed. We reduced the number of VSC to stabilize the paged pool memory consumption.
• The increase in paged pool memory consumption is seen in “nt!mm - Mm section object prototype ptes“, this can be viewed using poolmon.exe.
• We are seeing the issue on 4 file servers, it is acute on 2 file servers that have 4+ volumes.
• The server with the most trouble is a VMWare VM, 6 CPU cores, 16GB RAM, 4GB Page File, 6 VHDs (52TB total). When “nt!mm - Mm section object prototype ptes“ got to ~10GB the console becomes unresponsive until processes die or are killed to free RAM. The server shows no signs of stress other than high paged pool, it is coasting.
• The page file is not being used significantly so we believe this is an issue with memory allocation not memory utilization.

Memory Consumption

• The image shows % Committed Memory
• A: %Committed Memory increases while VSC are taken, and appears to never be released.
• B: %Committed Memory spikes during backups, and is released when backups complete.
• C: A reboot to release paged pool memory
• D: Memory consumption flattens after the VSC limit is reached

 .

Relevant Articles

• unable to allocate memory - Windows Server
• CodeMachine - Article - Prototype PTEs
• Getting to know the MmSt Pool Tag
• Pushing the Limits of Windows: Paged and Nonpaged Pool

Work Arounds

We have a few reasonable workarounds but any suggestions on the cause of this issue would be appreciated.

• Increment the VSC limit, allow paged pool to be consumed, reboot to clear paged pool, repeat (This is what we are currently doing).
• Move from taking VSCs every 1hr to taking VSCs every 2hr
• Temporarily increase commit limit by increasing page file and/or RAM, then resize and reboot

Let me know what you think!

Thanks,

Rob