Event ID 29 when starting KDC service on Windows Server 2008 R2 DC's

  • Article

I got the following escalation the other week:

We’re getting Event ID 29 on our new W2k8 R2 DC’s – our W2k3 DC’s in the same domain that do not get any error use Domain Controller Authentication certificates from the same SubCA and running certutil –verify –urlfetch <exported DC cert.cer> seems to verify all CDP locations successfully yet it returns error 0x80092013 (ERROR: Verifying leaf certificate revocation status returned The revocation function was unable to check revocation because the revocation server was offline).
Running the same certutil command on the same certificate but on a Windows Server 2003 DC returns success.

This turned out to be something of a puzzler – In general, event ID 29 is logged by the KDC service on a Domain Controller if the service either doesn't find any suitable domain controller certificate or if all the suitable certificates fail revocation checking.
Certutil is a good indicator about the POV of the DC with regards to revocation status (as long as you run it in the System context using psexec -s) and in this case we observed via tracing that all CDP locations were in fact being retrieved successfully and cached on both W2k3 and W2k8 R2.

The fact that the entire certificate chain was being verified successfully on W2k3 also meant that there shouldn’t be any issues with the chain itself.

….or did it? (da-da-daaaa +cue dramatic music) :)

We looked at several different factors here and discovered the following:

  • The RootCA cert the SubCA chained up to was from a fairly old OpenSSL implementation and was issuing a V1 CRL

  • As the CRL was V1 it was missing both an AKI and a CRL number (both are non-fatal for a V1 or V2 CRL)

  • The issued SubCA certificate was missing an AIA (also non-fatal as long as the SubCA cert is already present in the local store of the DC doing the verification)

  • There is a change in how WinHTTP authenticates between W2k3 and W2k8 as described in KB946401, by default no credentials are sent to servers outside of the domain (not a factor here as downloads of all CRL’s were also succeeding)

  • If Delta CRL’s are in use and the CDP is pointing to an IIS 7 server *or* an IIS 5.x server with URLScan installed then Double Escaping needs to be enabled (not a factor here as the Delta CRL’s were also being downloaded fine)

  • The SubCA certificate only had the Certificate Signing(04 - CERT_KEY_CERT_SIGN_KEY_USAGE) flag defined in the Key Usage extension (non-RFC5280 compliant)

 

A SubCA typically has the following Key Usage defined for it:

Digital Signature, Certificate Signing, Off-line CRL Signing, CRL Signing (86)

(CERT_DIGITAL_SIGNATURE_KEY_USAGE |CERT_KEY_CERT_SIGN_KEY_USAGE | CERT_OFFLINE_CRL_SIGN_KEY_USAGE |CERT_CRL_SIGN_KEY_USAGE)

In this case we were in fact looking at improvements that have been made in the Windows 2008 R2 code – on Windows Server 2003 the CRL check succeeds even if the SubCA cert used to sign the CRL doesn’t contain the appropriate flag. On Windows Server 2008 R2 it fails (as it should).

Turning on CAPI2 Operational logging on the DC’s and recycling the KDC service revealed the following Event ID 42 being logged that confirmed this:

<EventID>42</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>42</Task>
<Opcode>0</Opcode>
<Keywords>0x4000000000000005</Keywords>
<TimeCreated SystemTime='2011-09-09T08:46:12.730110300Z'/>
<EventRecordID>38669</EventRecordID><Correlation/><Execution ProcessID='536' ThreadID='6752'/>
<Channel>Microsoft-Windows-CAPI2/Operational</Channel>
<Computer>dc01.contoso.com</Computer>
<Security UserID='S-1-5-18'/></System>
<UserData><CertRejectedRevocationInfo>
<SubjectCertificate fileRef='2FAD4D75FA88B5025DE25F1CF2AD443C08F87B4B.cer' subjectName=' dc01.contoso.com' />
<IssuerCertificate fileRef='E7CD77ACC96868CDE3E456CFF221B77722A2C943.cer' subjectName='Test Contoso Internal Intermediate CA Smart Card'/>
<CertificateRevocationList location='Wire' url='http://crl.extranet.contoso.com/CertEnroll/Test%20Contoso%20Internal%20Intermediate%20CA%20Smart%20Card(3).crl' fileRef='AA12197D39683C69922609B946547185F6BAE382.crl' issuerName='Test Contoso Internal Intermediate CA Smart Card'/>
<Action name='IsCrlSignKeyUsagePresent'>
<Error value='800B0106'>A certificate being used for a purpose other than the ones specified by its CA.
</Error></Action>
<EventAuxInfo ProcessName='lsass.exe'/>
<CorrelationAuxInfo TaskId='{94862B6E-B8C8-4DF5-B7D3-DECC4ADB7C3F}' SeqNumber='27'/>
</CertRejectedRevocationInfo>
</UserData>
</Event>

 

I.e. the revocation check was failing because the SubCA cert didn’t have the flag that authorized it to sign a CRL. 

W2k3 and XP just don’t check this flag when verifying the CRL so it only becomes a factor on Vista+ clients and servers.

We also determined that the KDC service on Windows Server 2003 is hard-coded to accept the KDC certificate even if it encounters CERT_TRUST_IS_OFFLINE_REVOCATION errors during the revocation checking that is performed when the KDC service starts.

In comparison, the KDC service on Windows Server 2008 R2 can be configured to behave in the same way as W2k3 DC's by creating and setting the DWORD registry entry UseCachedCRLOnlyAndIgnoreRevocationUnknownErrors under HKLM\SYSTEM\CurrentControlSet\services\kdc \ to a non-zero value.

Note that setting this does not make the KDC accept a revoked certificate – if the certificate has been revoked it will still be rejected (assuming the CRL that contains the revocation can be downloaded by the server).

Further details.

Description of the changes to network retrieval of PKI objects in Windows Vista Service Pack 1 and in Windows Server 2008
http://support.microsoft.com/kb/946401

Smartcard authentication changes [in Windows 2008]
http://technet.microsoft.com/en-us/library/cc721959(WS.10).aspx

How to avoid Delta CRL download errors on Windows Server 2008 with IIS7
http://blogs.technet.com/b/pki/archive/2008/02/25/how-to-avoid-delta-crl-download-errors-on-windows-server-2008-with-iis7.aspx

Updated requirements for a Windows Server 2008 R2 domain controller certificate from a 3rd party CA 
http://social.technet.microsoft.com/wiki/contents/articles/updated-requirements-for-a-windows-server-2008-r2-domain-controller-certificate-from-a-3rd-party-ca.aspx