February 2009
Volume 24 Number 02
Stack Trace - What Are These System PTEs?
By Bob Golding | February 2009
Contents
What System APIs Are Used to Map These Addresses?
Stop 3F Track PTEs Are Not Always Needed
Kernel Stacks
System PTEs, like all PTEs, represent addresses in the system address space. What makes these PTEs different is that they are a system resource. There are only so many per system. These "system PTEs" are "dynamic slots" in the system address space map, which means that a range of addresses are reserved in the system address space and are used to map pages on demand and assign an address to a buffer, or they are used for system thread stacks. The address range lives between nonpaged pool and paged pool. Below is from mi386.h.
E1000000 | Start of paged system area | | Kernel mode access only. | | | | | +------------------------------------+ | | | System PTE area - for mapping | | kernel thread stacks and
MDLs | | that require system VAs. |<----- Range reserved for dynamic mapping. | Kernel mode access only. | | | +------------------------------------+ | | | NonPaged System area | | Kernel mode access only. | | | +------------------------------------+ FFBE0000 | Crash Dump Driver area | | Kernel mode access only. | +------------------------------------+
So since its size is between paged and nonpaged pool, adjustments to these memory pools have an affect on the size of the mapping window. Also, since this is part of the address space, factors that affect the size of the memory pools (that is, /3GB) affect the size of the mapping window.
What System APIs Are Used to Map These Addresses?
Some APIs that map these addresses are MiMapLockedPagesInUserSpace, MiUnmapLockedPagesInUserSpace, MmMapLockedPages, and MmUnmapLockedPages. These APIs map a buffer into the dynamic address range. They will map the entire buffer, so the buffer size determines how many contiguous pages get used.
As you can see, there is an unmap API for every map API. This is important to note. Buffers should be unmapped; otherwise, the mapping window will close. In other words: no more system PTEs. The more pages that get mapped the more system PTEs get used. As I mentioned before, system stacks use this range. Video does as well in order to map video buffers. So there must be enough left over to map buffers. When system PTEs get low, performance issues will occur. Also, a Stop 3F can occur, but these should be rare, as NT code is more tolerant of low resource conditions.
Stop 3F Track PTEs Are Not Always Needed
A lack of system PTEs is commonly troubleshot using TrackPtes to find the consumer of a high number of the PTEs. However, sometimes this is not useful because TrackPtes is not implemented by the sysptes.c code itself. So it's possible to reserve PTEs without being tracked. TrackPtes is only implemented by some of the callers of MiReserveSystemPtes.
Kernel Stacks
One common consumer of sysptes that is not tracked by TrackPtes is kernel stacks. Fortunately, the memory manager offers us several globals, which give insight into the sysptes being used to map kernel stacks.
In the example below, the memory manager has only created 23,000 sysptes. That's a fairly small number, but it's not uncommon with /3GB. Of these sysptes, 17,860 are being used to map kernel stacks. Analysis of the processes and threads did not find an apparent leak; the customer's application architecture required many GDI threads. After analyzing the machine's memory usage, we were able to tune the memory manager to increase the total system PTEs and make enough room for things to run.
1: kd> !sysptes System PTE Information Total System Ptes 23006 1: kd> dc nt!MmKernelStackPages l 1 80483680 000045c4 .E.. 1: kd> ?45c4 Evaluate expression:
17860 = 000045c4 1: kd> dc MmLargeStacks l 1 8048368c 0000037b 1: kd> dc MmSmallStacks l 1 8048367c 00000385 1: kd> ?37b*f Evaluate expression: 13365 = 00003435 1: kd> ?385*3 Evaluate expression: 2703 = 00000a8f 1: kd> ?3435+a8f+37b+385 Evaluate expression: 17860 = 000045c4
Bob Golding has been with Microsoft since 1997. He is a Senior Escalation Engineer on the Global Escalation Services team where he supports Microsoft's largest customers with their most critical issues.
David Butler has been with Microsoft since 2000. He is an Escalation Engineer on the Global Escalation Services team where he supports Microsoft's largest customers with their most critical issues.