Why don't my .NET Framework patches install?

I recently worked an issue with a customer who was rolling out some patches to over 50,000 computers across his enterprise using SMS. SMS reported that the patches successfully installed on all workstations, but my customer noticed a strange anomoly that he couldn't explain. Even though SMS reported a successful installation on all of his workstations, it also reported many of these same workstations as unpatched on successive scans. Something was obviously awry.

Upon investigation, we found a log file in the customer's temp directory that contained the following:

[2007-09-22 01:00:00] ProcessFiles completed.
[2007-09-22 01:00:06] PatchExec: Unhandled Exception Microsoft.WindowsInstaller.InstallerException: The configuration data for this product is corrupt. Contact your support personnel. 
    at Microsoft.WindowsInstaller.Installer.get_Products() 
    at Microsoft.DDPatch.PatchInstallation.ApplyPatchToInstalledProducts(String mspFile, String target, TargetProduct[] targetProducts, Boolean quiet, String logfile, String& lastPatchedProductName) 
    at Microsoft.DDPatch.PatchInstallation.Install(String targetProduct, Boolean quiet, String logfile) 
    at Microsoft.DDPatch.PatchExec.Main(String[] args)
[2007-09-22 01:00:15] Process returned: 0

This sure doesn't look right. Obviously something didn't go well during the patch installation, but the return code we see here is 0. No wonder SMS believed that the patch successfully installed.

What happened? This particular patch (and some other .NET Framework patches) consists of a Windows Installer patch file (a .msp file) along with some wrappers around some APIs for the Windows Installer. These wrappers provide, among other things, support for rollback and for proper installation of the patch. In this particular case, one wrapper called an API that enumerated products installed by the Windows Installer. That API enumerated the HKEY_CLASSES_ROOT/Installer/Products registry key which contains a GUID for each product that's installed, but in my customer's case, it also contained some textual string values that were written to it by a bad installer from a third-party company.

When the API encountered this bad registry key, it returned an error code to our wrapper. Our wrapper then wrote out the failure to the log file and returned 0 because it had successfully done its job and logged the failure. Unfortunately, the other wrapper got the return code of 0 and believed that the install was successful.

It's worth pointing out that this is an extremely uncommon scenario. The keys written to the HKEY_CLASSES_ROOT/Installer/Products registry key should be added using the Windows Installer API. The Windows Installer API will create valid GUID keys and all will be fine. However, not all installation packages play by the rules, and in this case, we got bit.

WARNING: If you encounter a problem like this, it's important that you not just extract the .msp file out of the patch and install it manually. If you do that, you'll lose the ability to fully roll back the patch. Instead, you should try and locate the offending registry key and remove it.

In the long term, you won't have to worry about this. We've fixed this problem in our wrappers in order to provide a more fail-safe installation experience. The new wrappers should be incorporated into our .NET Framework patches soon.