Stand alone unit test programs

VS macro to insert GUIDs

Real Team Foundation command line help

Changing console fonts

Subtle visible whitespace in VS2005

Visual Studio column guides redux

Reading custom configuration sections in .NET 2.0

SxS problems

Turning off Vista problem reporting

Reader/writer lock in .NET 3.5

Putting SxS errors in your face

Running MSBuild scripts programmatically

VS macro to insert TODO:

Listing all solution project properties

Getting accurate per thread timing on Windows

Recently we decided to give everyone on the Visual Studio Diagnostics team a Director's title (minus the actual promotion).  Mine was "Director of Random Technologies" probably on account of the fact that I have a tendency to act a bit like Hammy in "Over the Hedge" when it comes to software and computers.  It's all good.

The realization hit me this year that the blogs I read the most are the ones that are focused in some way in their scope.  They have a theme.  Sadly my blog has thus far had no theme.  It also helps when trying to write blog entries if you have a theme, otherwise it's a bit like a book without a plot. 

So I've decided refocus, and to use my MSDN blog to write solely about Microsoft diagnostic tools and technologies.  We'll see how it works out.

Another thing I'm going to try and do is keep my entries short and sweet.

The trace profiler works by rewriting the method bodies of your program to include probes.  A probe is a small piece of code, usually a function call, that calls out to the profilers collection logic so that it can record the current location of the program execution.

The subject of this blog is how those probes get into your methods.  This process is called instrumentation.  I'll discuss two approaches, and then I'll discuss the pros and cons of each.

Static Instrumentation

In this approach the instrumentation occurs before your program even runs.  Essentially the probes are baked in to you binary.  There are two ways this can be done.  One is to have the compiler put the probes in.  Visual C++ takes this approach to implent Profile Guided Optimizations (PGO).  Another is to do what is called post link instrumentation.  With this approach you take the binary that has been emitted by the compiler, crack it open to find all the methods, instrument and then rewrite the binary like nothing ever happened. 

Pros & Cons:
This approach yields much faster start-up times.  The images that is loaded in to memory is the image that runs.

The rewrite process destroys .NET strong naming, and thus images must be resigned after instrumentation, or verification skipping must be enabled.  Statically instrumented images can also be placed in the native image cache.

Rewriting the images means you have to deal with the whole question of where to put the files that you instrument.  This can be quite a housekeeping chore for the user.

Rewriting PE files correctly is relatively straightforward for .NET applications.  It's really hard for native applications, particularly for x64 images, where instrumenting methods can turn them from leaf methods to non-leaf methods.

Dynamic Instrumentation. 

In this approach the instrumentation occurs as the program is running.  In the case of a just in time compilation environment such as .NET, this is acheived by using the .NET profiling API to get notified when methods are about to be compiled and executed for the first time.  Your profiler gets the opportunity to rewrite the method that is about to be run, by adding probes or whatever you want.  Then the program continues on.  Dynamic instrumentation can also be performed on native images too, but doing this is hard and rarely done.  The approaches you could take to pull this involve injecting a DLL into the process, suspending it and then using debug information to patch and rewrite methods.

Pros & Cons

Yields significantly slower start-up times and increased memory usage because each method body must be reallocated, as well as the the some CPU load to instrument on the fly. 

If you want to do instrumentation at the line level, this approach requires loading additional data from the .pdb files in at runtime too.  If you load the actual .pdb's that has a huge runtime cost. 

Because the original file was not modified, .NET strong naming is not broken.  However, profiling of images in the native image cache must be disabled, as these cannot be dynamically instrumented because they are already compiled to native code

Conclusion 

Hopefully this entry has given you an overview of the two approaches.  Please comment with any question you may have.

<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi" xmlns:netfx="http://schemas.microsoft.com/wix/NetFxExtension">
  <Product 
    Id="*" 
    ...
    UpgradeCode="35ED4D41-BA5F-4598-8266-A1B9EB512345">
    ...
    <Upgrade Id="35ED4D41-BA5F-4598-8266-A1B9EB512345">
      <UpgradeVersion Minimum="$(var.ProductVersion)" OnlyDetect="yes" Property="NEWERVERSIONDETECTED"/>
      <UpgradeVersion Minimum="1.0.0" IncludeMinimum="yes" Maximum="$(var.ProductVersion)" IncludeMaximum="no" Property="OLDERVERSIONBEINGUPGRADED"/>
    </Upgrade>
    ...
    <InstallExecuteSequence>
      <RemoveExistingProducts After="InstallInitialize" />
    </InstallExecuteSequence>

Note the product's Id attribute set to "*" so you don't forget to change it each time, but make sure the UpgradeCode stays the same forever.  You could change the IncludeMaximum attribute to "yes" on the second UpgradeVersion element for debug builds so that you can continuously reinstall during development, even though your version number may not be changing.  However I don't recommend you ship that way, because you want repair existing installations by default not uninstall/install them.

My thanks to Bob Arnson on the WiX team for providing this tip.

It's hard to click on something that flashes up and disappears in the blink of an eye. There's two reasons that I can think of for showing this balloon at all. One is as an informational message to educate you about macro usage. The other is a security feature so that you know that a macro is running. Neither of which I find that compelling.

A few minutes work with Regmon and one can discover that setting the following value in the registry to one will disable this balloon for ever more.

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\8.0]
"DontShowMacrosBalloon"=dword:00000006

Craig Skibo, one of the Microsoft experts on the VSSDK say that this security balloon was the subject of much debate, and that it is in fact coded to not show up after 5 or 6 times of being clicked on. Nice touch, if you could actually click on it. Hence the the value of 6 above.

Sometime last year, I was reading a page in the MSDN documentation when a certain line caught my attention. The page was "Locating the Assembly through Codebases or Probing" and the line was this one, at the very bottom of the page:

"If Assembly2 is not found at either of those locations, the runtime queries the Windows Installer."

For some reason, this line struck me as interesting – a hitherto unseen mystery in the .NET runtime. OK, it's not like I was finding some hidden clue in the Hypernotomachia Poliphili or anything. But this is my world, and I take my kicks where I can get 'em.

I'd read this line many times before, but I'd never really thought anything of it. Now I was suddenly realizing in all the times I'd seen assembly probing fail, I'd never ever seen it report that it was asking the Windows Installer to find the assembly. Was this just a documentation error, or was this a real feature of the .NET runtime?

To test things out, I created a simple application. It basically consisted of a managed executable (A) that loaded a library (B) using Assembly.Load(AssemblyName). Trying to do things the fastest way possible, I created a Windows installer by creating a Setup Project in VS 2003. I installed the application, deleted library B and ran A, and sure enough there was an assembly load exception, but there was no message in the Fusion log about the Windows Installer being invoked to find the assembly.

I sent of a couple of e-mails to Microsoft folks, and got a response back from the .NET team that indicated that if all was well with the installation, a call to MsiProvideAssembly with the correct parameters should cause the missing assembly to reappear. OK, so .NET must be relying on this call internally to find the assembly, right? I ran the installed application under a debugger, and I did not see msi.dll, the Windows Installer library, being loaded by the .NET runtime. Very strange. A quick peek confirmed that any Windows Installer probing code had been stripped out of the Rotor sources, so no help there.

So I logged a bug at the Feedback center. It even got a mention in Junfeng Zhangs blog, one of the CLR loader wizards. The problem, it would appear, was with the installer built by the VS 2003 Setup project. It was not doing the right magic when it installed the application to allow .NET to find the assembly. What that magic was, and why .NET never loaded msi.dll remained a mystery.Well, I did eventually figure out what was going on, but I never got around to blogging about it.

So for the other pathologically curious folks out there, here's the answer to the mystery. Firstly, as we've already established, .NET is not really at fault. It is however up to some of its usual sneakiness in the never ending quest for performance, which is what made it hard for me to point the finger at the actual cause of the problem in the first place. The fact is, the .NET runtime doesn't even bother loading msi.dll unless it can be certain that the cost will be worthwhile. In order to do that, it actually does some of the checks that MsiProvideAssembly would do. That's why I never saw msi.dll being loaded. To discover this I ran Regmon on the installed application (with the library deleted), on the assumption that that was the most likely place for further clues, and searched for the word "Installer" in the output. It turns out that the runtime scans the following list of registry keys for a record of the missing assembly:

HKLM\SOFTWARE\Microsoft\Windows CurrentVersion\Installer\Managed\<SID>  Installer\Assemblies\Global\<Strong Name>
HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer Managed\<SID>\Installer\Assemblies\<File Path>HKCU\Software\Microsoft\Installer\Assemblies Global\<Strong Name>
HKCU\Software\Microsoft\Installer\Assemblies\<File Path>HKLM\SOFTWARE\Classes\Installer\Assemblies\Global\<Strong Name>
HKLM\SOFTWARE\Classes\Installer\Assemblies\<File Path>\

If any record of the assembly being installed is found, the runtime loads msi.dll to bring back the missing assembly. <SID> appears to be the security identifier of the user that the process is running as. <File Path> is the full path to the file, but any '\' is substituted with a ''. <Strong Name> is the full assembly strong name. "Global" refers to the Global Assembly Cache (GAC).

Unfortunately, in the case that the assembly is found the Windows Installer obviously scans the same registry keys again looking for the assembly. I won't hide the fact that I don't like these sorts of shenanigans – by making assumptions about the behavior of the Windows Installer, the Windows Installer team is now essentially prevented from making changes to the way that MsiProvideAssemby works without a corresponding update to the .NET runtime. But what's done is done.

So how do you create an installer that will cause assembly probing to query the Windows Installer and bring back missing assemblies? Well, I'm sure that one of the commercial installers will do the right thing, but my preferred solution is just to use WiX, given that it's free.

So, let's build another sample from scratch. Firstly, we'll create an assembly called Binding.exe. Here's the code:

using System;
using System.Reflection;
using System.Globalization;
[assembly: AssemblyVersion("1.0.0.0")]
[assembly: AssemblyKeyFile("binding.snk")]
namespace Binding
{
 class MainClass
 {
  static void Main(string[] args)
  {
   AssemblyName myAssemblyName = new AssemblyName();
   myAssemblyName.Name = "ClassLibrary1";
   myAssemblyName.Version = new Version("1.0.0.0");
   myAssemblyName.SetPublicKeyToken(new byte[]
    {0x32, 0xea, 0x68, 0xe6, 0x16, 0x01, 0x2a, 0xe3});
   myAssemblyName.CultureInfo = new CultureInfo("");
   Assembly ass = Assembly.Load(myAssemblyName);
   MethodInfo method =
    ass.GetType("ClassLibrary1.Class1").GetMethod("Method1");
   method.Invoke(null, null);
  }
 }
}

Just to make things interesting this code builds an AssemblyName and loads an assembly explicitly. Note that everything should be strongly named in order for things to work properly. Don't forget to set the CultureInfo. The code for ClassLibrary1.dll is just:

using System;
using System.Reflection;
using ClassLibrary2;

[assembly: AssemblyVersion("1.0.0.0")]
[assembly: AssemblyKeyFile("binding.snk")]

namespace ClassLibrary1
{
  public class Class1
  {
    public Class1()
    {
    }
   
    public static void Method1()
    {
      Console.WriteLine("ClassLibrary1.Class1.Method1");
      Class2.Method2();
    }
  }
}

ClassLibrary1 has an implicit dependency on the assembly ClassLibrary2.dll. The code for ClassLibrary2.dll is:

using System;
using System.Reflection;
using ClassLibrary2;[assembly: AssemblyVersion("1.0.0.0")]
[assembly: AssemblyKeyFile("binding.snk")]namespace ClassLibrary1
{
  public class Class1
  {
    public Class1()
    {
    }
    
    public static void Method1()
    {
      Console.WriteLine("ClassLibrary1.Class1.Method1");
      Class2.Method2();
    }
  }
}

OK, now for the installer code:

<?xml version="1.0" encoding="utf-8"?>
  <Wix xmlns=http://schemas.microsoft.com/wix/2003/01/wi>
  <Product Id="DEB8151D-1EEB-4549-A5AD-405C4CD28085" Name="BindingProduct" Language="1033" Version="0.0.0.0" Manufacturer="Microsoft Corporation">
  <Package Id="511F5323-27B5-47C1-A488-8F92ED38EFEB" Description="Binding Test Product" Comments="Used for testing assembly binding" InstallerVersion="200" Compressed="yes" />
  <Media Id="1" Cabinet="PRODUCT.CAB" EmbedCab="yes" />
  <Directory Id="TARGETDIR" Name="SourceDir">
    <Directory Id="ProgramFilesFolder" Name="PFiles">
      <Directory Id="BindingProductDirectory" Name="ProdDir" LongName="BindingTest">
        <Component Id="BindingComponent" Guid="3A505CEA-C035-4F4C-A7CD-C0CAE01F492F">
          <File Id="BindingExeConfigFile" Name="BINDIN01.CON" LongName="Binding.exe.config" DiskId="1" src="Binding.exe.config" mce_src="Binding.exe.config"/>
          <File Id="BindingExeFile" Name="BINDING.EXE" LongName="Binding.exe" DiskId="1" src="Binding.exe" mce_src="Binding.exe" Assembly=".net" KeyPath="yes" AssemblyApplication="BindingExeConfigFile" AssemblyManifest="BindingExeFile" Vital="yes"/>
        </Component>
        <Component Id="ClassLibrary1Component" Guid="F6E5ECF8-EBD8-42C1-8BC0-9A3E7CAF985C">
          <File Id="ClassLibrary1DllFile" Name="CLASSL01.DLL" LongName="ClassLibrary1.dll" DiskId="1" src="ClassLibrary1.dll" mce_src="ClassLibrary1.dll" Assembly=".net" KeyPath="yes" AssemblyApplication="BindingExeConfigFile" AssemblyManifest="BindingExeFile" Vital="yes"/>
        </Component>
        <Component Id="ClassLibrary2Component" Guid="FCEAC462-3710-4A01-ACAB-B41F522ACD41">
          <File Id="ClassLibrary2DllFile" Name="CLASSL02.DLL" LongName="ClassLibrary2.dll" DiskId="1" src="ClassLibrary2.dll" mce_src="ClassLibrary2.dll" Assembly=".net" KeyPath="yes" AssemblyApplication="BindingExeConfigFile" AssemblyManifest="BindingExeFile" Vital="yes"/>
        </Component>
      </Directory>
    </Directory>
  </Directory>
  <Feature Id="BindingProductFeature" Title="Binding Test Product Feature" Level="1">
    <ComponentRef Id="BindingComponent" />
    <ComponentRef Id="ClassLibrary1Component"/>
    <ComponentRef Id="ClassLibrary2Component"/>
  </Feature>
  </Product>
</Wix>

The key component of the installer is the use of the AssemblyApplication attribute on the File element. This attribute points to the Binding.exe.config file for the application. This is the file that .NET will look for to determine if your application is installed before calling the Windows Installer bring back any missing assemblies.

Compile the three assemblies, then the installer. Run the installer. No navigate to the folder C:\Program Files\BindingTest\. Run Binding.exe. It outputs the two method names. Now for the magic. Delete ClassLibrary1.dll and ClassLibrary2.dll. Go on, don't be shy. Done that? Now run Binding.exe again. Instead of an AssemblyLoadException the program should run as if nothing had happened. Check again and the two DLL's are magically back. A self repairing application. Pretty cool, eh?

Lastly, I built a small program to test things out called CheckInstalled.cs, as follows:

using System;
using System.IO;
using System.Runtime.InteropServices;
using System.Reflection;
using System.Text;

// After running msiexec /i WixSetup.msi the command:
//
// CheckInstalled ClassLibrary1 "C:\Program Files\BindingTest\Binding.exe"
//
// Should return the path to ClassLibrary1.dll. See one of the registry keys:
//
// HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\Managed\
//   <SID>\Installer\Assemblies\Global\<Strong Name>
// HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\Managed\
//   <SID>\Installer\Assemblies\<File Path>
// HKCU\Software\Microsoft\Installer\Assemblies\Global\<Strong Name>
// HKCU\Software\Microsoft\Installer\Assemblies\<File Path>
// HKLM\SOFTWARE\Classes\Installer\Assemblies\Global\<Strong Name>
// HKLM\SOFTWARE\Classes\Installer\Assemblies\<File Path>

class MainClass
{
  [DllImport("msi.dll", CharSet=CharSet.Auto, SetLastError=true)]
  static extern uint MsiProvideAssembly(
    string assembly, string context, int mode, int info,
    System.Text.StringBuilder path, ref int pathSize); 
 
  public static void Main(String[] args)
  {
    if (args.Length < 1)
    {
      Console.WriteLine(
        "USAGE: {0} <assemblyName> [<installPath>]",
        Path.GetFileName(Assembly.GetExecutingAssembly().Location));
      Console.WriteLine(
        "\nAssembly name is just a full or partial assembly name. " +
        "The <installPath> is case sensitive");
      return;
    }
    
    if (args.Length > 1)
      Console.WriteLine(
        "Checking for installation of assembly '{0}' at install path '{1}'",
        args[0], args[1]);
    else
      Console.WriteLine("Checking for installation of assembly '{0}'",
        args[0]);
  
    try
    {
      Console.WriteLine(EnsureAssemblyIsInstalled(
        args[0], args.Length < 2 ? null : args[1]));
    }
    catch (SystemException e)
    {
      Console.WriteLine("ERROR: " + e.Message);
    }
  }
 
  public static string EnsureAssemblyIsInstalled(
    string assemblyName, string installPath)
  {
    int buffer = 1024;
    StringBuilder path = new StringBuilder(1024);  
    uint code = MsiProvideAssembly(assemblyName, installPath,
      0, 0, path, ref buffer);    
    if (code != 0)
      throw new SystemException("Windows Installer error " +
        code.ToString());   
      return "Found '" + path.ToString() + "'";
  }
}

Run this program as follows:

CheckInstalled ClassLibrary1 "C:\Program Files\BindingTest\Binding.exe.config"

It should report that ClassLibrary1 was found. Delete it and run the application again and you'll see Windows Installer bring the file back, this time with a user interface. This UI is suppressed by the .NET runtime.

So next time you see something hiding underneath a rock, go and check it out. It might be nothingit could be a nasty smelly bug, or it could just be a hidden gem.

I recently upgraded some old unit tests to see what was involved, and it was really straightforward. First I'll make some assumptions. You have a VS 2003 solution that you've recently upgraded to VS 2005 and which contains some NUnit test projects. I'll also assume you are using C# as your development language.

Here's a rundown of what I did:

1. Add a new Test Project to your solution (Test menu -> New Test -> Unit Test). This is going to add a Solution Items folder containing .vsmdi and .testrunconfig files. It should look something like the image on the below in your Solution Explorer.
   
   
2. I usually delete the AuthoringTests.txt and ManualTest.mht files, the Additional Test Attributes #region, and all the // TODO: and /// comments.
3. Drag all your .cs files from your old NUnit project. This will copy them into the VSUnit project.
4. Back-up your old NUnit project somewhere, then remove the project from the solution and delete or rename the project directory on disk.
5. At this point you should also rename your new VSUnit project. If you right click Rename it, bear in mind that that it sadly won't update the directory name. You'll have to close the solution, rename the directory in Explorer and open the solution .sln file in Notepad and change the directory before TestProject1.csproj.
6. When you're done with the tidy-up you are ready to start converting the test code. The diagram below shows the basic conversions. Just follow the conversions, 1 through 7. li>When you're done with the tidy-up you are ready to start converting the test code. The diagram below shows the basic conversions. Just follow the conversions, 1 through 7.
   
   
   Note that VSUnit also has support for assembly wide setup and cleanup which you might want to make use of. But just focus on the conversion for now. Note the change to a static method for the class setup and cleanup, and the addition of the TestContext parameter.
7. When you're done with step 6, try to compile your project. If it compiles, you are ready to run it and see what happens! Select the test then Test -> Start Selected Test Project Without Debugger.

Note that VSUnit also has support for assembly wide setup and cleanup which you might want to make use of. But just focus on the conversion for now. Note the change to a static method for the class setup and cleanup, and the addition of the TestContext parameter.

When you're done with step 6, try to compile your project. If it compiles, you are ready to run it and see what happens! Select the test then Test -> Start Selected Test Project Without Debugger.

You're probably wondering about all those Asserts in your NUnit tests. The really good news is that the Assert class exists in VSUnit and the methods signatures and behavior are by in large the same. The compiler will tell you where there are problems.

Other things to note. You can debug your unit tests just like with NUnit by setting breakpoints in the unit test and then selecting Test -> Start Selected Test Project With Debugger. The TestContext parameter also lets you do some really cool stuff when setting up and configuring your tests.

You can take an existing set of unit tests and easily run the methods in specific orders by adding one or more Ordered Tests to your unit test project. The only way to do this in NUnit was to alphabetize your test method names!

There's lots of other fun stuff available too. My favorite feature by far is to turn on coverage profiling for the tests, by going to Test -> Edit Test Configurations then select Coverage and select the assemblies you want to check the coverage for. I think you'll soon agree that these two tools were made for each other.

There's a problem with doing instrumented profiling of applications in Visual Studio 2005 Team System that comes up when working with strongly named assemblies. The problem is quite simply that instrumentation breaks the strong name signature of your assemblies. And in working around this problem, I'll show you how you can actually be hiding start-up performance problems in your .NET applications.

Let us recap how assembly signing works in .NET. First, you create a public/private encryption key pair using sn.exe -k or similar. Then you take your assemblies files and you sign them. This signing can be done with the sn.exe tool or more transparently by using the /keyfile or /keycontainer command line options of the C# or VB compilers.

Basically what the signing process actually does is to compute a cryptographic hash from the metadata, intermedate language (IL) and other parts of your assembly file. It then takes this hash and encrypts it using the private key from your key pair. Finally this signature blob and your public key are embedded in the assembly. Now you can send your assembly out into the world safe in the knowledge that no one can mess with it.

Any CLR that your assembly gets presented to can now validate that your assembly has not been tampered with. This is done by first calculating the same hash that the signing process calculated. Then, the CLR takes the signature blob from the assembly and uses the public key to reverse the encryption performed by your private key. This will only work if the public key is truly the partner of the private key used to encrypt the signature. Hence the importance of keeping the private key, uh, private. If all is well, the decrypted signature and the calculated hash will match, and Bob's your Uncle!

Note: The 16 character public key token that you see as part of the full name of a strongly named assembly is not the signature hash referred to above. It is just a hash of your assemblies public key which makes the key a little easier to work with. For example, the public key token

7e16da7b2e1bcfa6

is merely a hash of the public key

0024000004800000940000000602000000240000525341310004000001000100e91eb19e092741
e85e7eda996ba566d491be9a300b2c26a8ce015398c87529f76075385c960bbc4dcf24ece0d1ad
a9f312dd0a42538d490be9228148645ecf5f45aadc12404af4c07d4b8060056b4724174ecc7d65
c65ac455a7321f71adfa8be981b02e9c5bd721d588395fa212c4a79cc007cf3a60738a0777614e
15b990a5

Which would you rather type?

So signing allows tampering to be caught, but it doesn't prevent it. And there's a back door into the signing process using the sn tool that allows signature verification to be turned off for assemblies containing a specific public key. In VSTS performance (and coverage) instrumentation works by inserting additional calls at the basic block boundaries in your applications IL and then rewriting the application binaries to disk. We call this static instrumentation because it's done before the program runs. Thus it breaks strong names signatures by changing the hash that will be computed when the assembly is verified by the CLR.

OK, big deal right? What's all this got to do with performance profiling? Well, the issue is that validating assembly signatures takes time. All that cryptographic stuff is computationally expensive. So, if you turn off verification for an assembly using something like:

sn -Vr *,b5f241dcf0cb58c4

Then you are potentially hiding a performance problem that might occur when assemblies are fully signed and signing is not disabled. Now, before we go running screaming into the hills, there's one caveat that has to do with the the global assembly cache (GAC). Putting an assembly in the GAC requires that the assembly be strongly named. The full signature verification for assemblies in the GAC is actually only performed when the assembly is inserted into the GAC, and not on subsequent loads of the assembly. So in that sense, there's actually a small performance boost to be had by putting assemblies in the GAC. But all other strongly named assemblies will incur a verification check the first time they are loaded into an AppDomain.

For what types of applications might turning off strong naming incur an artificial performance boost? Well, in practice this is only likely to be a problem for applications that depend on very large numbers of strongly named private assemblies, and even then it's only going to skew the applications start-up time. Applications that load and unload assemblies via additional AppDomains might also incur some penalty, but then I suspect that other factors such as memory fragmentation and disk I/O are much more likely to cause performance noise that will hide this issue.

So, what is test signing? Test signing is a halfway point between fully signed/fully verified and unsigned/unverified. It's a new feature in .NET 2.0. With test signing you can create a throw away test key that you can use to sign your assemblies. The CLR verification process then goes through the full set of computations to verify the assembly hash against the test public key, resulting in the same performance hit as if the assembly were fully signed. However, the assembly doesn't actually contain the public key for the test key pair, but the real public key for the assembly. In this way the strong name of the assembly is not broken. The public key for the test key pair actually lives in the registry.

OK, time for an example. Let's say you just generated a new project called ConsoleApplication1. Let's say that it starts of fully signed using your companies real key pair. Now we instrument:

>vsinstr ConsoleApplication2.exe
Microsoft (R) VSInstr Post-Link Instrumentation 8.0.50727.0
Copyright (C) Microsoft Corp. All rights reserved.

File to Process:
   C:\Projects\ConsoleApplication2\bin\Debug\ConsoleApplication2.exe
Original file backed up to C:\Projects\ConsoleApplication2\bin\Debug\ConsoleApplication2.exe.orig

Warning VSP2001 : C:\Projects\ConsoleApplication2\bin\Debug\ConsoleApplication2.exe is a strongly
named assembly.  It will need to be re-signed before it can be executed.
Successfully instrumented file C:\Projects\ConsoleApplication2\bin\Debug\ConsoleApplication2.exe.
Warning VSP2013 : Instrumenting this image requires it to run as a 32-bit process.  The CLR header
flags have been updated to reflect this.

Note that the message about having to sign the assembly before it can be executed, should really say something like "before it can be strong name verified". Now you create your test key and test sign the assembly.

>sn -k TestKey.snk
Microsoft (R) .NET Framework Strong Name Utility  Version 2.0.50727.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Key pair written to TestKey.snk

And you'll need just the public key too, so lets get that now:

>sn -p TestKey.snk TestPublicKey.snk.
Microsoft (R) .NET Framework Strong Name Utility.  Version 2.0.50727.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Public key written to TestPublicKey.snk

You need the public key token that's currently in the assembly, which remember is from the real public key pair and is part of the real strong name of the assembly.

>sn -T ConsoleApplication2.exe
Microsoft (R) .NET Framework Strong Name Utility  Version 2.0.50727.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Public key token is b5f241dcf0cb58c4

Now you do the test signing:

>sn -TS ConsoleApplication2.exe TestKey.snk
Microsoft (R) .NET Framework Strong Name Utility  Version 2.0.50727.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Assembly 'ConsoleApplication2.exe' successfully re-signed

And finally you register the test signing public key:

>sn -Vr *,b5f241dcf0cb58c4 TestPublicKey.snk
Microsoft (R) .NET Framework Strong Name Utility  Version 2.0.50727.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Verification entry added for assembly '*,b5f241dcf0cb58c4'

Sadly the message that the last step gives you is identical to the one you get for regular verification skipping registration. Go to the registry and look under:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\StrongName\Verification\*,b5f241dcf0cb58c4

(or whatever your public key token is). You'll see a TestPublicKey value containing the public key token.

Now you can run your performance tests and measure the real cost of checking all those assembly strong name signatures.