How does VS 2005 Embed Native Manifest Files

  • Article

With VC 2005 the native libraries such as the CRT, MFC, and ATL can now be installed as Side by Side assemblies. These assemblies are installed in the WinSxS directory. This is similar to the GAC but for native code. However when this happens your assemblies that load these libraries will need a manifest. Visual Studio will do this for you by default and if you are doing a Makefile build then you can make the appropriate changes. Nikola (PM for Visual C++) did a great job of outlining the changes that need to be made and what the overall process is doing in his blog:

The actual MSDN topic is listed at:

One note about the MSDN topic on what Visual Studio does (Manifest Generation in Visual Studio) is that it does not provide any differentiation between the build process with or without incremental linking enabled. The process it describes occurs when incremental linking is enabled, however the process is different without incremental linking. Nikola’s blog entry (Why I see ''Embedding manifest...'' message in Output window?) does make the distinction and provides some clarity.

For me it was interesting to understand the different techniques that were used to embed the manifest files and to truly understand this I stepped through each of the command lines. For a Debug build the process goes roughly something like this:

  1. Compiler compiles the application and generates the *.obj files.
  2. An empty manifest file is generated if this is a clean build and if not the previous one is reused.
  3. The resource compiler (rc.exe) compiles the *.manifest file to a *.res file.
  4. Linker generates the binary (EXE or DLL) with the /incremental switch and embeds the dummy manifest file. The linker also generates the real manifest file based on the binaries that your binary depends on.
  5. The manifest tool (mt.exe) is then used to generate the final manifest.
  6. The resource compiler is invoked one more time.
  7. Finally, the Linker does another incremental link, but since the only thing that has changed is the *.res file that contains the manifest it is a short link.

How do these steps map to the actual command lines used to create your binary. The build log that Visual studio generates is a great place to start.

The compiler steps are normal so I will skip those. This leads us to the resource compiler which is generating the resource file from our initial manifest:

rc.exe /fo".\Debug\ManifestTest.exe.embed.manifest.res" "Debug\resource.txt"

The file resource.txt contains:

1 /* CREATEPROCESS_MANIFEST_RESOURCE_ID */ 24 /* RT_MANIFEST */ ".\\Debug\\ManifestTest.exe.embed.manifest"

NOTE:

  • 1 – This is the resource ID of the Manifest for an EXE
  • 2 – This would be the resource ID of the Manifest for a DLL

After the res file is generated the linker is then invoked to generate our binary:

link.exe
/OUT:"Debug\ManifestTest.exe"
/INCREMENTAL
/MANIFEST
/MANIFESTFILE:"Debug\ManifestTest.exe.intermediate.manifest"
/DEBUG
/PDB:"debug\ManifestTest.pdb"
/SUBSYSTEM:CONSOLE
/MACHINE:X86
kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib
".\Debug\ManifestTest.obj"
".\Debug\stdafx.obj"
".\Debug\ManifestTest.exe.embed.manifest.res"

I highlighted the interesting pieces of the command line in bold. The first one (/INCREMENTAL) enables us to avoid a complete re-link with each minor change. However if something (mt.exe for instance) edits the binary then the linker would be forced to do a full link next time and /INCREMENTAL would no longer help us shorten build time. The next two switches (/MANIFEST and /MANIFESTFILE) tell the linker where to output the manifest file. The final option at the end (".\Debug\ManifestTest.exe.embed.manifest.res") is embedding the resource file that we created. Once the link is complete we will have 2 files that are important:

  1. ManifestTest.exe.embed.manifest – The original manifest that we first link into the EXE.
  2. ManifestTest.exe.intermediate.manifest – This file is generated by the linker and will contain all of the dependencies as determined by the linker.

Now it is time to invoke mt.exe:

mt.exe /out:".\Debug\ManifestTest.exe.embed.manifest" /notify_update /manifest ".\Debug\ManifestTest.exe.intermediate.manifest"

This will process the manifest that was generated by the linker and then will overwrite the file that we started out with if the 2 files are different. You will notice that even though this command is executed each time the output file is only updated if there is a change. Visual Studio will determine if ManifestTest.exe.embed.manifest has changed from the version that we used to generate the original resource. If it has not we stop right here because we used the correct file for the original link. If it has changed then we proceed.

Assuming that the manifest has been updated the next setup is the regenerate the resource file and for this we use the exact same command line.

The final step is to re-link our binary:

link.exe
/OUT:"Debug\ManifestTest.exe"
/INCREMENTAL
/MANIFEST
/MANIFESTFILE:"Debug\ManifestTest.exe.intermediate.manifest"
/DEBUG
/PDB:"debug\ManifestTest.pdb"
/SUBSYSTEM:CONSOLE
/MACHINE:X86
kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib
".\Debug\ManifestTest.obj"
".\Debug\stdafx.obj"
".\Debug\ManifestTest.exe.embed.manifest.res"

This command line is identical to the one that we used above but the only file that has changed is the RES file so the linker will only update the resource part of the binary.

Why would the manifest file change?

Above we mention that we do not bother to move forward if the manifest file is unchanged and that begs the question, what would change the manifest file. There are a couple of possibilities that I know of:

  1. You updated your dependency versions – For instance if you install VS 2005 SP1 there will probably be an update to the CRT files and so if you build after the update you will take a dependency on the newer version. Since the manifest file contains the version of the file you depend on it would get updated.
  2. Take a new dependency – If your project begins depending on a new file that is loaded from the WinSxS folder using the fusion load then the manifest will be updated.

So, it is pretty safe to say that the manifest will be updated infrequently. However moving forward more and more data will be pushed into the manifest file so it may change more frequently in the future.

Now, that we have looked at the process when increment linking is enabled, what happens if it is not enabled? The process is shorter and the method for embedding the manifest in the binary is different. Before you leveraged an incremental link if the file had changed, in this case we do:

  1. Compiler compiles the application and generates the *.obj files.
  2. Linker generates the binary (EXE or DLL) and the real manifest file based on the binaries that your binary depends on.
  3. The manifest tool (mt.exe) is then used to generate the final manifest and embeds it directly into the binary.

The linker command line is:

Link.exe
/OUT:"Release\ManifestTest.exe"
/INCREMENTAL:NO
/MANIFEST
/MANIFESTFILE:"Release\ManifestTest.exe.intermediate.manifest"
/PDB:"release\ManifestTest.pdb"
/SUBSYSTEM:CONSOLE
/OPT:REF
/OPT:ICF
/LTCG
/MACHINE:X86
kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib
".\Release\ManifestTest.obj"
".\Release\stdafx.obj"

You may have noticed that we are not linking a resource file in at all. This is because the mt.exe tool will take care of this:

mt.exe /outputresource:"..\release\ManifestTest.exe;#1" /manifest ".\Release\ManifestTest.exe.intermediate.manifest"

When we invoke mt.exe we use the /outputresource instead of /out in the previous example. This enables mt.exe to embed the resource directly into the binary that the linker just generated. This is great because as you can see we avoid several of the steps above. We are still using the number 1 since this an EXE manifest. If we did a DLL manifest it would be 2.

One step that is common to both processes that we discuss is the create of mt.dep:

@echo Manifest resource last updated at %TIME% on %DATE% > ".\Debug\mt.dep"

This will simply provide a timestamp for the last we updated the manifest resource. The output looks like:

Manifest resource last updated at 23:59:03.65 on Sun 05/21/2006

After manually walking through these steps I felt that I had a much better handle on this process and nice idea about what Visual Studio was doing behind the scenes!