TMG Web Listener Certificate "Private Key handle error" 0x80090016

  • Article

You may face an issue with a certificate assigned to a listener that suddenly becomes invalid and therefore the incoming SSL connection are dropped.

Restarting the service you will show the following error:

Event Source: Microsoft Firewall
Event ID: 14060
Description: Description: Cannot load an application filter Web Proxy Filter ({4CB7513E-220E-4C20-815A-B67BAA295FF4}). FilterInit failed with code 0x80092004.
To attempt to activate this application filter again, stop and restart the Firewall service.

The problem can be caused by the permission on private keys of the certificate store becoming corrupted. This may be affecting one or more certificates.
In these cases deleting the bad certificate and re-importing can help to resolve the the problem most of the times.

Find more information in this article: http://blogs.technet.com/b/isablog/archive/2009/03/10/unable-to-start-microsoft-firewall-service-in-isa-server-2006.aspx

In some cases you may have lost the original PFX file or forgot the password and need the fix the issue using a different approach.
In this article we will discuss how to better diagnose the issue and try to fix it, this may or may not work in your environment depending on the entity of the damage.

You can identify the invalid certificates by opening the TMG console, even if the Firewall service is not running, and try to assign the right certificate to all of your listeners.
By unselecting the checkbox “Show only valid certificates”, you will see a message similar to that in the screenshot below:

In the properties of the listener, when selecting a certificate, you may get the status “Private key handle error” or “Invalid key”

You can try fixing the issue from the Certificates console:

  • Execute MMC

  • Add the Certificate snap-in for the Local computer

  • In the Personal store right click on the certificate, All task, Manage Private keys

  • If you can assign Full control to the local Administrators group and to SYSTEM

  • Then go back to the TMG console and select the certificate, it should appear valid

  • Save and apply the configuration and try to start the Firewall service

However you may be unable to assign the permission from the certificates console, you may get an Access denied error.
In this case you will have to identify the file with the certificate’s private key, the file is located in the folder c:\ProgramData\Microsoft\Crypto\RSA\MachineKeys

To troubleshoot the issue, you can use Process Monitor from SysInternals (DOWNLOAD: http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx).Once downloaded and extracted follow these steps:

  • Close all running programs, just keep the Certificates MMC open

  • Start Process Monitor, a capture will start automatically

  • On the Certificate MMC go to Manage Private keys as described above

  • Once you get the Access Denied error go back to Process Monitor and press CTRL-E to stop the capture

  • Click on Tools then Process Tree

  • Scroll down to mmc.exe, right click then Add process to Include filter, then click Close

  • The events will be filtered and only those generated by MMC will be displayed

  • Scroll down in the list and you will see some rows generated while trying to access to the private key file and ACESS DENIED in the result column, right click on any of them then Jump To …

  • The right folder and file will automatically be opened and you will be able to assign the permissions. You may also need to take ownership of the file in order to do that.

  • Then go back to the TMG console and select the certificate, it should appear valid

  • Save and apply the configuration and try to start the Firewall service

It is important to always re-select the certificate in the TMG console, this way the additional permissions required by the TMG Firewall service will be assigned automatically.

Depending on the entity of the damage you may need to follow the above steps for all the certificates affected by the issue.

Author:

Gianni Bragante
Support Engineer – Microsoft Forefront Edge Security Team

Reviewer:

Philipp Sand
Sr. Support Escalation Engineer – Microsoft Forefront Edge Security Team