Part 3: Memory Protection Technologies

Published: August 09, 2004 | Updated: September 15, 2004

By Starr Andersen, Technical Writer;Vincent Abella, Technical Editor

This document is Part 3 of “Changes to Functionality in Windows XP Service Pack 2” and provides detailed information about the memory protection technologies included in Windows XP Service Pack 2. You can obtain the other parts of the paper in the Microsoft Download Center, at https://go.microsoft.com/fwlink/?LinkId=28022.

This document applies to Microsoft® Windows® XP Service Pack 2 (SP2) for the 32-bit versions of Windows XP Professional and Windows XP Home Edition. It does not describe all of the changes that are included in the service pack, but instead highlights those changes that will have the most impact on your use of Windows XP SP2 and provides references to additional information that may be available.

Data Execution Prevention

What does data execution prevention do?

Data execution prevention (DEP) is a set of hardware and software technologies that perform additional checks on memory to help protect against malicious code exploits.  In Windows XP SP2, DEP is enforced by both hardware and software.

Hardware-enforced DEP

Hardware-enforced DEP marks all memory locations in a process as non-executable unless the location explicitly contains executable code. There is a class of attacks that attempt to insert and execute code from non-executable memory locations. DEP helps prevent these attacks by intercepting them and raising an exception.

Hardware-enforced DEP relies on processor hardware to mark memory with an attribute that indicates that code should not be executed from that memory. DEP functions on a per-virtual memory page basis, usually changing a bit in the page table entry (PTE) to mark the memory page.

The actual hardware implementation of DEP and marking of the virtual memory page varies by processor architecture. However, processors that support hardware-enforced DEP are capable of raising an exception when code is executed from a page marked with the appropriate attribute set.

Both Advanced Micro Devices™ (AMD) and Intel® Corporation have defined and shipped Windows-compatible architectures that are compatible with DEP.

Beginning with Windows XP Service Pack 2, the 32-bit version of Windows utilizes the no-execute page-protection (NX) processor feature as defined by AMD or the Execute Disable bit feature as defined by Intel. In order to use these processor features, the processor must be running in Physical Address Extension (PAE) mode. The 64-bit versions of Windows XP uses the NX processor feature on 64-bit extensions and certain values of the access rights page table entry (PTE) field on IPF processors.

It is hoped that all future 32-bit and 64-bit processors will provide support for hardware-enforced data execution prevention. Microsoft continues to work with processor vendors to encourage the adoption and development of DEP technologies.

Software-enforced DEP

An additional set of data execution prevention security checks have been added to Windows XP SP2. These checks, known as software-enforced DEP, are designed to mitigate exploits of exception handling mechanisms in Windows. Software-enforced DEP runs on any processor which is capable of running Windows XP SP2. By default, software-enforced DEP only protects limited system binaries, regardless of the hardware-enforced DEP capabilities of the processor.

Who does this feature apply to?

Application and driver developers should be aware of data execution prevention and the requirements of software running on a supporting platform. Applications that perform just-in-time (JIT) code generation or execute memory from the default process stack or heap should pay careful attention to DEP requirements.

Driver developers are encouraged to be aware of PAE mode on platforms supporting hardware-enforced DEP. PAE mode behavior on Windows XP Service Pack 2 systems is changed to improve driver compatibility.

What new functionality is added to this feature in XP Service Pack 2?

Data execution prevention on 32-bit versions of Windows and applications

Detailed description

Hardware-enforced DE P

With few differences, the overall behavior of data execution prevention on Windows is the same for both 32-bit and 64-bit versions of Windows. To provide consistency for application and driver developers, the memory protection model (including DEP) is designed to be the same for both 32-bit and 64-bit versions of Windows.

Application developers should be aware of DEP behavior in user-mode. A user mode DEP exception results in a STATUS_ACCESS_VIOLATION (0xc0000005) on Windows systems. The first parameter of ExceptionInformation that is located inside the EXCEPTION_RECORD structure contains the type of access violation that occurred. A value of 8 for ExceptionInformation[0] indicates the access violation was an execution violation.

In most processes, the STATUS_ACCESS_VIOLATION exception will be an unhandled exception and result in termination of the process.

DEP is also applied to drivers in kernel mode. DEP for memory regions in kernel mode cannot be selectively enabled or disabled. On 32-bit versions of Windows, DEP is applied to the stack by default. This differs from kernel-mode DEP on 64-bit versions of Windows, where the stack, paged pool, and session pool have data execution prevention applied.

Device drivers are not permitted to execute code from the stack when DEP is enabled. A DEP access violation in kernel-mode will result in a bugcheck 0xFC: ATTEMPTED_EXECUTE_OF_NOEXECUTE_MEMORY

Software-enforced DEP

Software-enforced DEP performs additional checks on exception handling mechanisms in Windows. If the program’s image files are built with Safe Structured Exception Handling (SafeSEH), software-enforced DEP ensures that before an exception is dispatched, the exception handler is registered in the function table located within the image file.

If the program’s image files are not built with SafeSEH, software-enforced DEP ensures that before an exception is dispatched, the exception handler is located within a memory region marked as executable.

Why is this change important? What threats does it help mitigate?

The primary benefit of data execution prevention is that it helps to prevent code execution from data pages such as the default heap, various stacks and memory pools. In normal operations of the system, code is not typically executed from the default heap and stack. Hardware-enforced DEP detects code that is running from these locations and raises an exception when execution occurs. If the exception is unhandled, the process will be terminated. Execution of code from protected memory in kernel mode results in a bugcheck.

Although terminating a process or causing the system to fail with a bugcheck do not appear to be ideal experiences, this helps prevent malicious code from executing. Preventing malicious code from executing on the system may prevent damage to your system or propagation of malicious code whose harmful effects could easily exceed those of a process terminated by a bugcheck.

DEP can help prevent some exploits in which a virus or other attack tool has injected executable code into a process and then attempted to execute the injected code. On a system with DEP, execution of the injected code should result in an exception. Software-enforced DEP can help mitigate against exploits of exception handling mechanisms within Windows.

A secondary benefit of DEP relates to good engineering and best practices for application and driver developers. Data execution prevention forces developers to avoid executing code out of data pages without explicitly marking the pages as executable.

What works differently?

Application Compatibility

Some application behaviors are expected to be incompatible with data execution prevention. Applications which perform dynamic code generation (such as Just-In-Time code generation) and that do not explicitly mark generated code with Execute permission might have compatibility issues with data execution prevention. Applications which are not built with SafeSEH must have their exception handlers located in executable memory regions.

Applications that attempt to violate DEP will receive an exception with status code STATUS_ACCESS_VIOLATION (0xC0000005). If an application requires executable memory, it must explicitly set this attribute on the appropriate memory by specifying PAGE_EXECUTE, PAGE_EXECUTE_READ, PAGE_EXECUTE_READWRITE or PAGE_EXECUTE_WRITECOPY in the memory protection argument of the Virtual* memory allocation functions. Heap allocations using the malloc() and HeapAlloc() functions are non-executable.

Driver Compatibility

Driver compatibility issues with data execution prevention mostly center on PAE mode-induced compatibility issues.

Note   PAE is only required on systems with processors that support hardware-enforced DEP.

On its own, DEP might create compatibility issues with drivers that perform code generation or use other techniques to generate executable code in real time. Although many drivers with such behavior would have been fixed—as DEP is “always on” for drivers loaded on 64-bit versions of Windows—there is no guarantee that all drivers have been updated. However, there are few drivers that employ these techniques, and it is not expected that DEP alone will cause a large quantity of driver compatibility issues.  

The primary driver compatibility concern is running Physical Address Extension (PAE) mode on 32-bit systems. PAE mode enables processors to address greater than 4 gigabytes (GB) of memory. The primary difference between PAE memory paging and non-PAE memory paging schemes is the extra level of paging that is required in PAE mode (3 levels instead of 2).

Some drivers might fail to load if PAE is enabled, because the device might be unable to perform 64-bit addressing or the drivers might assume that PAE mode requires more than 4 GB of RAM. Such drivers expect that they will always receive 64-bit addresses when in PAE mode and that they (or their device) are incapable of interpreting the address.

Other drivers might load in PAE mode but cause system instability by directly modifying system page table entries (PTEs). These drivers expect 32-bit PTEs, but receive 64-bit PTEs in PAE mode instead.

The largest driver PAE compatibility issue involves direct memory access (DMA) transfers and map register allocation. Many devices that support DMA, usually 32-bit adapters, are not capable of performing 64-bit physical addressing. When run in 32-bit mode, the device can address all physical address space. In PAE mode, it is possible that data would be present at a physical address greater than 4 GB. To allow devices with these constraints to function in this scenario, The Windows 2000 Server family and later provide double-buffering for the DMA transaction by providing a 32-bit address that is indicated by a map register. The device can perform the DMA transaction to the 32-bit address and the kernel copies the memory to the 64-bit address that is provided to the driver.

When the system runs with PAE disabled, drivers for 32-bit devices never require their map registers to be backed by real memory. This means that double-buffering is not necessary, since all devices and drivers are contained within the 32-bit address space. Based on testing of drivers for 32-bit devices on 64-bit processor–based computers, it is expected that most client-tested, DMA-capable drivers expect unlimited map registers.

To constrain compatibility issues, Windows XP Service Pack 2 includes hardware abstraction layer (HAL) changes that mimic the 32-bit HAL DMA behavior. The altered HAL grants unlimited map registers when the system is running in PAE mode. In addition, the kernel memory manager ignores any physical address above 4 GB. Any system RAM beyond the 4 GB barrier would be made unaddressable by Windows and be unusable in the system. By limiting the address space to 4 GB, devices with 32-bit DMA bus master capability will not see a transaction with an address above the 4 GB barrier. Because these changes remove the need to double-buffer the transactions, they avoid a class of bugs in some drivers related to proper implementation of double buffering support.

As a result of these changes to the HAL and memory manager, the impact to device driver compatibility is expected to be minimal on systems running Windows XP Service Pack 2 with data execution prevention enabled.

System Compatibility

A final DEP compatibility concern derives from systems with PAE mode enabled, even though they may not be designed for more than 4GB of physical RAM. Microsoft has noticed in testing some systems with processors that support hardware-enforced DEP fail to boot or have other stability issues when the processor is running in PAE mode.  

PAE mode is a requirement for leveraging the NX processor feature. Therefore, system designers and firmware engineers should be aware that even though the system’s chipset and firmware may not have been designed to support more than 4GB of physical RAM, the system may be running in PAE mode.

Of particular concern is system firmware that interprets page table entries to determine instructions executed by the operating system. Page table entries are extended to 64 bits in length when the processor is running in PAE mode. System designers and firmware developers are encouraged to contact their processor and chipset vendors for more information on how to safely determine instructions executed by the operating system.

System designers working with AMD processors can obtain more information in the “BIOS and Kernel Developer’s Guide for AMD Athlon™ 64 and AMD Opteron Processors.” To obtain this paper, go to the AMD Athlon™ 64 Web site at https://go.microsoft.com/fwlink/?LinkId=28165 and click “BIOS and Kernel Developer’s Guide for AMD Athlon™ 64 and AMD Opteron Processors.”

Intel does not make detailed information about System Management Mode (SMM) available publicly. System designers working with Intel processors are encouraged to contact Intel directly for more information.

For more information regarding Windows support for PAE mode, see “Physical Address Extension - PAE Memory and Windows” on the Microsoft Web site at https://go.microsoft.com/fwlink/?LinkId=28166.

How do I resolve these issues?

Applications that require executable regions of memory must use the PAGE_EXECUTE, PAGE_EXECUTE_READ, PAGE_EXECUTE_READWRITE, or PAGE_EXECUTE_WRITECOPY attributes when allocating memory. Additionally, applications cannot execute from the default process heap or the stack. Most applications that perform actions incompatible with DEP will need to be updated to be compatible. Applications must also be built with SafeSEH or ensure their exception handlers are located in memory explicitly marked as executable.

An application can use the VirtualAlloc() application programming interface (API) to allocate executable memory with the appropriate memory protection options. At a minimum, the PAGE_EXECUTE memory protection option should be used. After the executable code has been generated, it is recommended that the application set memory protections to disallow write access to the allocated memory. Applications can disallow write access to allocated memory using the VirtualProtect() API. Disallowing write access ensures maximum protection for executable regions of process address space.

If a malicious process attempts to insert code into an executable region, the access would result in a STATUS_ACCESS_VIOLATION write exception. The application should attempt to make the executable regions of its address space as small as possible. This would result in a smaller attack surface through which executable memory could be injected into the process address space and be executed.

Additionally, sophisticated applications can control the layout of their virtual memory and create executable regions. These applications should attempt to locate executable regions in a lower memory space than non-executable regions. The purpose of locating executable regions below non-executable regions is to protect a buffer overflow from overflowing into executable memory.

A small number of executables and libraries may contain executable code in a data section of the image file. In some cases, applications may place small segments of code (commonly referred to as thunks) in the data sections. However, Data Execution Prevention will mark sections of the image file loaded in memory as non-executable unless the section has the executable attribute applied.

Therefore, executable code in data sections should be migrated to a code section, or the data section containing the executable code should be explicitly marked as executable. The executable attribute, IMAGE_SCN_MEM_EXECUTE (0x20000000), should be added to the Characteristics field of the corresponding section header for sections which contain executable code.

The Microsoft linker that is distributed with Microsoft VisualStudio products can add the executable attribute to a section using the /SECTION linker option. The /SECTION linker option has the following format:

/SECTION: Name ,[E][R][W][S][D][K][L][P][X][,ALIGN=#]

The E value indicates the executable attribute (0x20000000). More information about the /SECTION and other Microsoft linker options is available on the MSDN Web site at https://go.microsoft.com/fwlink/?LinkId=28167.

Additionally, the Microsoft COFF Binary File Editor (Editbin.exe) utility can be used to change the section attributes of an existing image. The Editbin utility has a /SECTION option with the following format:

/SECTION: Name[=newname][,[[!]{CDEIKOMPRSUW}][A{1248PTSX}]]

The E and C values indicate code and executable attributes respectively.  For more information about the Editbin utility and the /SECTION option, see the MSDN Web site at https://go.microsoft.com/fwlink/?LinkId=28168.

Microsoft plans to update Microsoft .NET Framework v1.0 and v1.1 to take advantage of DEP in Windows XP SP2. Applications that are dependent upon the Microsoft .NET Framework will continue to function normally, but will not benefit from DEP if it is enabled.

Microsoft encourages application developers that redistribute the Microsoft .NET Framework to update to Microsoft .NET Framework v1.0 Service Pack 3 or v1.1 Service Pack 1, which takes advantage of DEP, when it is available.

What settings are added or changed in Windows XP Service Pack 2?

System-wide Configuration of DEP

Data execution prevention configuration for the system is controlled through Boot.ini switches. Additionally, changes to System in Control Panel have been made to enable end users to easily configure DEP settings if they are logged onto the system as an administrator.

Windows supports four system-wide configurations for both hardware-enforced and software-enforced DEP.

Configuration	Description
OptIn (default configuration)	On systems with processors capable of hardware-enforced DEP, DEP is enabled by default for limited system binaries and applications that “opt-in,” With this option, only Windows system binaries are covered by DEP by default.
OptOut	DEP is enabled by default for all processes. Users can manually create a list of specific applications which do not have DEP applied using System in Control Panel. IT Pros and Independent Software Vendors (ISVs) can use the Application Compatibility Toolkit to opt-out one or more applications from DEP protection. System Compatibility Fixes (“shims”) for DEP do take effect.
AlwaysOn	This provides full DEP coverage for the entire system.  All processes always run with DEP applied. The exceptions list for exempting specific applications from DEP protection is not available. System Compatibility Fixes (“shims”) for DEP do not take effect. Applications which have been opted-out using the Application Compatibility Toolkit run with DEP applied.
AlwaysOff	This does not provide any DEP coverage for any part of the system, regardless of hardware DEP support. The processor will run in PAE mode with 32-bit versions of Windows unless the /NOPAE option is also present in the boot entry.

Hardware-enforced and software-enforced DEP are configured in the same manner. If the system-wide DEP policy is set to Opt-In, the same Windows core binaries and applications will be protected by both hardware and software-enforced DEP. If the system is not capable of hardware-enforced DEP, the Windows core binaries and applications will only be protected by software-enforced DEP.

Similarly, if the system-wide DEP policy is set to Opt-Out, applications which have been exempted from DEP protection will be exempted from both hardware and software-enforced DEP.

The four system-wide DEP configurations are controlled through boot.ini switches. The Boot.ini settings are as follows:

/noexecute= policy_level

where policy_level is defined as AlwaysOn, AlwaysOff, OptIn, or OptOut.

Any existing /noexecute setting in the Boot.ini file is not changed when Windows XP SP2 is installed or if a Windows operating system image is moved across computers with and without hardware-enforced DEP support.

During installation of Windows XP SP2, the OptIn policy level is enabled by default unless a different policy level is specified in an unattended installation. If the **/noexecute=**policy_level setting is not present in the boot entry for a version of Windows which supports DEP, the behavior is the same as if the /noexecute=OptIn option was included.

End users who are logged on as administrators can manually configure DEP between the OptIn and OptOut policies using the Data Execution Prevention tab inside the System Properties dialog box. The following procedure describes how to manually configure DEP on the computer:

1. Click Start, click Control Panel, and then double-click System.

2. Click the Advanced tab. Then, under Performance, click Settings.

3. Click the Data Execution Prevention tab.

4. Click Turn on DEP for essential Windows programs and services only to select the OptIn policy.

5. Click Turn on DEP for all programs and services except those I select to select the OptOut policy.

6. If you selected the OptOut policy, click Add and add the applications that you do not want to use DEP with.

IT professionals can control system-wide DEP configuration with a variety of methods. The Boot.ini file can be modified directly with scripting mechanisms or with the Bootcfg.exe tool which is included as part of Windows XP SP2.

For unattended installations of Windows XP SP2, you can use the Unattend.txt file to pre-populate a specific DEP configuration. You can use the OSLoadOptionsVar entry in the [Data] section of the Unattend.txt file to specify a system-wide DEP configuration.

Per-application DEP Configuration

For the purposes of application compatibility when DEP is set to the OptOut policy level, it is possible to selectively disable DEP for individual 32-bit applications.

For end users, the Data Execution Prevention tab in System Properties can be used to selectively disable DEP for an application.

For IT professionals, a new application compatibility fix named DisableNX is included with Windows XP Service Pack 2. The DisableNX compatibility fix disables data execution prevention for the program it is applied to.

The DisableNX compatibility fix can be applied to an application by using the Application Compatibility Toolkit. For more information about Windows application compatibility, see “Windows Application Compatibility” on the Microsoft Web site at https://go.microsoft.com/fwlink/?LinkId=23302.