How to detect programmatically if the system is paging

Question

How to detect programmatically if the system is paging

Sedlmair, Martin (SHS DI CT R&D CTC SA) 6

We are using memory mapped files to an extent where we produce data faster than the memory manager is able to write modified pages to the paging file. Ultimately this results in a stalling system where the user is not able to work anymore. What we want to do is to throttle our processing and data generation while the system is paging data to disk.

The problem is, that there is no signal or event (to our knowledge) that we can detect to see if we need to throttle.

Is there any signal/event for paging?

2 answers

Your answer

Answer 1

Limitless Technology 44,751

Hello there,

You can use the GlobalMemoryStatusEx function to determine how much memory your application can allocate without severely impacting other applications.

The information returned by the GlobalMemoryStatusEx function is volatile. There is no guarantee that two sequential calls to this function will return the same information.

The ullAvailPhys member of the MEMORYSTATUSEX structure at lpBuffer includes memory for all NUMA nodes.

https://learn.microsoft.com/en-us/windows/win32/api/sysinfoapi/nf-sysinfoapi-globalmemorystatusex

Hope this resolves your Query !!

--If the reply is helpful, please Upvote and Accept it as an answer–

Sedlmair, Martin (SHS DI CT R&D CTC SA) 6 Reputation points

2023-03-07T15:53:14.5+00:00

Hello, thanks for your answer, we already now about the GlobalMemoryStatus etc. but we do not get a signal when paging is active on windows. For example I can set a process working set limit using SedtProcessWorkSetSizeEx with a hard limit, lets say half of the available RAM. Then paging will be active if the consumed memory reaches 50% RAM. Still GlobalMemoryStatusEx returns a 50% memory load without telling me if paging is active.

Answer 2

Gary Nebbett 6,216

Hello Martin,

The only way I can think of tracking paging I/O is to use a real-time ETW (Event Tracing for Windows) session to trace file I/O; the read and write events include "flags" information that will have the IRP_PAGING_IO bit set (and the IRP_NOCACHE bit set too) for paging reads/writes.

Personally, I doubt that monitoring paging I/O specifically would be a good design decision. If your software can be throttled without any major drawbacks, then a more general test of the impact on system responsiveness would probably be easier to implement. One could also consider whether locking memory (e.g. via VirtualLock) could reduce the amount of paging caused by your application.

If SHS means what I think, it might be worthwhile talking to Alois Kraus.

Gary

Sedlmair, Martin (SHS DI CT R&D CTC SA) 6 Reputation points

2023-03-11T14:52:25.71+00:00

@Gary Nebbett : Thanks for your response. Yes, ETW is the last resort. To be a bit more specific to my needs.

I cannot use VirtualLock because I really want the system to page out things, because I need more memory ;-)

I'm currently doing paging myself with my own memory management, but I wanted to give memory mapped files a try, because in the end this is what I currently do myself. But with my home-made solution I exatcly know when I'm doing paging, so currently its easy to throttle and the whole process works stable.

When using MM-files my problem is that if I run into this low memory conditions the system completely stalls and I cannot check for anything - the system really stands still (no mouse movement, nothing and it seemed for a long time that windows haltet until I have waited for a very long time until windows recovers and the memory is in a better condition. What my current approach is to monitor the paging disk for activity (using a performance counter) and throttle the application accordingly, but of course there's more on the disk happing than only paging, so I would also throttle if someone is copying things to or from the disk.

BTW: Alois is my colleague we are already in discussions but haven't found a solution so far. There also was once a performance counter for memory mapped files:

https://learn.microsoft.com/en-us/windows-server/management/windows-performance-monitor/memory-performance-counter-mapped-file-bytes-written-sec

But the link has gone and I couldn't find an updated link.
Gary Nebbett 6,216 Reputation points

2023-03-11T15:37:19.5333333+00:00

Hello Martin,

A quicker way (compared to performance counters, which the documentation says should only be read once per second) to get that information is to use ZwQuerySystemInformation with a SYSTEM_INFORMATION_CLASS value of SystemPerformanceInformation. A structure describing the information returned can be found in the minidumpapiset.h include file (MINIDUMP_SYSTEM_PERFORMANCE_INFORMATION); the reason for the naming is that the results of retrieving the information can be included in suitably created minidump file stream of type SystemMemoryInfoStream.

The useful structure members for your purpose include DirtyPagesWriteCount, DirtyWriteIoCount, MappedPagesWriteCount and MappedWriteIoCount.

There may be other mechanisms that might help (e.g. Using IRP Priority Hints, also described in Chapter 6 of Windows Internals 7th edition) and I would still recommend using ETW and lots of patience to better understand in detail what causes the system stalls.

Gary
Sedlmair, Martin (SHS DI CT R&D CTC SA) 6 Reputation points

2023-03-11T15:42:14.25+00:00

@Gary Nebbett : Hmmm. I was looking at ZwQuerySystemInformation, but obviously not too deep ;-) I will take a look and dig a bit deeper. Thanks for helping out on the weekend.
Gary Nebbett 6,216 Reputation points

2023-03-11T15:46:34.29+00:00

Hello Martin,

I am retired - every day is like a weekend day :-)

Gary
Gary Nebbett 6,216 Reputation points

2023-03-11T17:43:56.8466667+00:00

Hello Martin,

Having thought some more about what you have written, I have an additional suggestion - but it depends upon the skills/resources that you have in-house.

Regardless of how hard you are hammering the file system, I would not expect the "stalls" that you have described. I would try enabling/using Forcing a system crash from the keyboard during such a stall and seeing if the crash dump (if one can be created) gives any insight into the causes of the "stall".

Gary
Sedlmair, Martin (SHS DI CT R&D CTC SA) 6 Reputation points

2023-03-13T11:43:54.2733333+00:00

Hi Gary,

thanks for the suggestion. In the meantime I have recreated my old minimal sample just to show you the problem, because you said you wouldn't expect the stalls. If you are interested you can see the example here:

https://github.com/msedi/MMFWindowsStall

Martin
Gary Nebbett 6,216 Reputation points

2023-03-13T12:07:39.9566667+00:00

Hello Martin,

I will see what tests I can usefully perform, but I will probably have to fire-up an old desktop computer for the purpose - the 150 GB (140 GiB) test file won't fit on my laptop!

Gary
Sedlmair, Martin (SHS DI CT R&D CTC SA) 6 Reputation points

2023-03-13T13:09:16.1666667+00:00

Hi Gary,

you don't necessarily need so much space. The problem occurs when you exceed the main RAM. On my desktop I have 96GB. But if you only have 32GB or even 16GB then it's sufficient to have size of 40GB or so.

Take care, the effect is worse on an HDD. Its better on an SSD.
Gary Nebbett 6,216 Reputation points

2023-03-13T16:06:43.66+00:00
Hello Martin,

One thing that you may not have considered is that the program does a lot of unnecessary (not useful) I/O. I have not pushed the program to a level where it starts to need the help of the modified page writer yet. An ETW trace of the I/O looks like this:

The stack at the time of the reads looks like this:

0xFFFFF80207CD00A5=ntoskrnl!IofCallDriver+0x55 0xFFFFF80207CB352C=ntoskrnl!IoPageReadEx+0x46C 0xFFFFF80207CDAB07=ntoskrnl!MiIssueHardFaultIo+0x107 0xFFFFF80207CDA0E7=ntoskrnl!MiIssueHardFault+0x207 0xFFFFF80207C576EA=ntoskrnl!MmAccessFault+0x31A 0xFFFFF80207E39F29=ntoskrnl!KiPageFault+0x369

The "empty" pages of the newly extended file are paged in.

Gary
Sedlmair, Martin (SHS DI CT R&D CTC SA) 6 Reputation points

2023-03-13T16:42:13.4333333+00:00

Hi Gary,

I'm not exactly sure what you mean by unnecessary I/O, but you made an interesting finding, I can see there are page reads where I do not expect them. For example, I see no reason why there is a PageRead and where in the code the PageRead is requested. Nevertheless thanks for looking in it. If I find more time I will also record some ETW traces to dig deeper.
Gary Nebbett 6,216 Reputation points

2023-03-13T17:07:31.1933333+00:00

Hello Martin,

The "not useful" I/O are the "page in" reads of the "empty" pages in the newly created-and-extended output file - your test program is reading 150 GB of zero bytes (not useful, at least in the context of the test program) and writing 150 GB of "output" data (also zeroes in the test case). Since some of the reads and writes will be interleaved (when the modified page writer is active) then that will, in addition, lead to a lot of "seeking" on a classic HDD.

Gary

Share via

How to detect programmatically if the system is paging

2 answers

Your answer