NXCOMPAT and the C# compiler
The C# compiler in Visual Studio 2008 and the .NET 3.5 Framework (csc.exe) is now generating PE files with the NXCOMPAT bit set. What is that bit and who cares, you ask? You may very well care if your application interops with native binaries or exposes a plugin model to 3rd parties. First, some background...
DEP is short for Data Execution Prevention. It is a technology that exists in Microsoft operating systems which prevents execution of code from memory pages which are not marked as executable. DEP exists to reduce the attack surface available to malicious software that is trying to hijack a process, and it has been acknowledged to be very helpful in that regard. In Windows Vista, the set of processes and applications to which DEP is applied is configurable by administrators, but there is also a role for application developers.
In the header of a PE file there is a flag called IMAGE_DLLCHARACTERISTICS_NX_COMPAT. This flag affects whether or not the OS enables DEP for a process. Setting this flag tells the OS that the image is compatible with DEP. For executable images, if this flag is set, the process is run with DEP enabled unless the machine is configured with the DEP policy set to AlwaysOff. If the image is a DLL and the flag is set, the OS skips checking the DLL against a compatibility database which results in a small performance improvement. All of this applies to x86, 32-bit processes only. On a 64-bit OS, DEP is always enabled for 64-bit processes, but 32-bit processes are configured by the PE flag and system policy as described above. So how does one control the flag in the PE header?
Since the C# compiler emits PE files which are MSIL only and therefore compatible with DEP, the output binaries from the VS 2008 and .NET 3.5 C# compilers have this flag set. Our expectation is that the vast majority of C# executables produced by these compilers will be part of a DEP-compatible application. For that reason we did not surface a compiler switch to configure the NXCOMPAT setting. Of course you can write a C# application that uses a native or mixed binary which is not compatible with DEP. Some ATL types in 7.1 and earlier used to do simple code generation into data pages which is a DEP no-no. If your application is generating IP_ON_HEAP exceptions, then you may need to clear the IMAGE_DLLCHARACTERISTICS_NX_COMPAT flag for your executable. To do this you can use EDITBIN.EXE from the VC toolset like so:
editbin.exe /NXCOMPAT:NO <your binary>
If you're using Visual Studio, you can add a post build step to your executable's project. You'll need to setup the environment so that EDITBIN's dependencies can be resolved. Since the post build steps you author in Visual Studio's properties page are written into a batch file that is launched by the build process, you can use Visual Studio's VSVARS32.BAT to establish the right environment. My post build step looks like this:
call $(DevEnvDir)..\tools\vsvars32.bat
editbin.exe /NXCOMPAT:NO $(TargetPath)
If you sign the binary in Visual Studio, flipping the IMAGE_DLLCHARACTERISTICS_NX_COMPAT flag in the post build step after the binary has been signed will result in an assembly that will fail strong name validation. To work around this you'll need to begin signing your binary as part of the post build steps. To do this, use SN.EXE from the Windows SDK.
The .NET 2.0 and VS 2005 compilers are also affected
We like DEP so much that when you install .NET Framework 2.0 SP1 your C# compilers in VS 2005 and .NET Framework 2.0 will also begin to emit binaries with the IMAGE_DLLCHARACTERISTICS_NX_COMPAT bit set. This will undoubtedly surprise a few developers...download a framework service pack, recompile, run your app, and you're now getting IP_ON_HEAP exceptions. Obviously this is not ideal, but aggressively building a computing ecosystem filled with DEP-enabled applications and their accompanying security benefits is very beneficial to Windows users. If you begin to encounter IP_ON_HEAP exceptions after installing .NET Framework 2.0 SP1, you can use the same technique described above to clear the IMAGE_DLLCHARACTERISTICS_NX_COMPAT bit in VS 2005. The only difference I'm aware of is that the SDK (and therefore the location of SN.EXE) has moved.
Comments
Anonymous
December 13, 2007
The comment has been removedAnonymous
December 14, 2007
To complicate matters further, there are 2 versions of sn.exe - 32bit and 64bit. The 64bit one AFAIK isn't shipped with the NDP. It comes as part of the Windows SDK.Anonymous
January 02, 2008
Welcome to the thirty-eighth Community Convergence. These posts are designed to keep you in touch withAnonymous
January 15, 2008
The comment has been removedAnonymous
January 18, 2008
I forgot to say what really made me mad about this NXCOMPAT thing: It's wrong! It's a bug! Writing a file with the NXCOMPAT flag, without making any effort to verify that the file is actually NX-compatible, is a bug. It may be politically correct, but it isn't any less of a bug than marking all exe's as '64-bit code' because "64 bits is twice as cool!" Investing effort to implement a bug, and then shipping it?? What were you thinking?Anonymous
January 20, 2008
在VS 2008中使用非托管DLL的时候，可能出现内存位置访问无效的错误，文章介绍了我找寻到的解决方案Anonymous
January 21, 2008
We are now busy tracking down DEP errors (from within the .NET library) in OUR application (i.e. we are now testing Microsoft's code in our application) just because we now have .NET SP1 installed on our development computer. Our (just) released application was built just before .NET SP1 magically appeared on our build computer so our current customers are SAFE for now. However, I just discovered our build computer has been updated; Magically our application has now been blessed with NXCOMPAT; it will now fail (IN the MICROSOFT .NET LIBRARY NOT OUR CODE). Good thing people have started blogging about this (and we found this one) before I inadvertently (WITHOUT KNOWLEDGE) released a update to our application with potential bugs in it. Thanks for NOT providing a compiler switch so I can add even MORE lines to our Postbuild.cmd process. Not a happy camper.Anonymous
January 31, 2008
The comment has been removedAnonymous
February 21, 2008
The comment has been removedAnonymous
April 24, 2008
Hi, So far I haven't been bit by this so I think this is funny because not executing data pages has been built into Intel chips since the 80386 and on into the current Pentium chips. This was a "feature" of the old CTOS operating system when running in "protected mode". The problem was Windows 32 bit mode didn't take advantage of it, choosing instead to use all 32bits for a memory address instead of using the segment descriptor and offset method needed for protected mode.Anonymous
August 19, 2008
I tried to run it on my exe, and it worked fine. Then I tried to include it in the post-build event like you said: call $(DevEnvDir)..toolsvsvars32.bat editbin.exe /NXCOMPAT:NO $(TargetPath) This gave the following error when I build it: The command "call C:Program FilesMicrosoft Visual Studio 9.0Common7IDE..toolsvsvars32.bat editbin.exe /NXCOMPAT:NO D:WorkDotNetRadiusStudio.NETRadiusStudiobinReleaseStudioVision.exe" exited with code 9009. This is also a problem inside Visual Studio, because it is a GUI control that fails to load in the forms designer. Could this be solved by running EDITBIN.EXE on devenv.exe?Anonymous
September 18, 2008
The comment has been removedAnonymous
September 22, 2008
The comment has been removedAnonymous
October 16, 2008
Hi All, I've got a similar issue. I've got a bug with by COM Interop and by using the following command in my build it get rids of the issue: call $(DevEnvDir)..toolsvsvars32.bat editbin.exe /NXCOMPAT:NO $(TargetPath) I've got it to work without signing my assemblies with a strong Name. However when we release our software we need to sign our assemblies using a .snk file. I've added the above code to my post build event however i can no longer run my app since it fails with Strong Validation Failed. What i need to do is call the above code and then sign my .exe file. How do i do this through the post build events? I'm currently using the Signing tab in visual studio 9 in my projects properties to sign my assemblies. Is there any other way to do this via the post build event? Have seen the following command but can not get it to work because i dont know what to put for MyModule.netmodule parameter al /out:MyAssembly.dll MyModule.netmodule /keyfile:sgKey.snk Any ideas? Any suggestions will be help. Thanks in advance. Regards NikAnonymous
November 20, 2008
I added this to my post-build event on a VB project. call "$(DevEnvDir)....VCbinvcvars32.bat" call "$(DevEnvDir)....VCbineditbin.exe" /NXCOMPAT:NO "$(TargetPath)" It executes successfully, but if the "primary output" of that project is included in a deployment project, the exe that gets included in the output MSI does NOT have DEP disabled. I have found that targetpath points to .....projectname\binx86Releaseprojectname.exe by experimentation, I have also found that if I edit bin ......projectnamebinx86Release......objx86releaseprojectname.exe that is, the obj folder instead of the bin folder then the exe that gets included in the MSI DOES have DEP disabled. I'm a happy camper, my problem is resolved, but I am curious as to why. Is it that the deployment project pulls the output from the wrong folder, or is it some timing issue about when things are copied from obj to bin?Anonymous
November 20, 2008
I should have added that I am using Visual Studio 2008Anonymous
December 09, 2008
I am using VS 2005 and Mappoint Active X control, when i build the application on Vista i am getting the error Mappoint Failed to Load. The i have added the call "$(DevEnvDir)....VCbinvcvars32.bat" call "$(DevEnvDir)....VCbineditbin.exe" /NXCOMPAT:NO "$(TargetPath)" and executed the application, this time Mappoint is working fine. When i deploy same project with ClickOnce deployment, i am getting Mappoint Failed to load on Vista Desktop. Please suggest me, how can i solve this problem.Anonymous
August 25, 2009
I got a mappoint failed to load error. What should i include in the "$(TargetPath")" . What should this be pointing to?Anonymous
August 29, 2011
Is Ed Maurer beiing facetitious when he says they love it so much!!?? Talk of "seemless backward compatiblity"Anonymous
November 14, 2013
Hi , I have a legacy COM dll built in VC++6.0 Could you please let me know if there is a way to make it DEP compatible in Visual studio 6.0? Please let me know. Your help in this is greatly appreciated. Thanks, Vinod.P.K