This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Hello!
Deployed the Windows Hello Hybrid Azure AD infrastructure joined with Key Trust.
Any user in the on-premises enterprise AD environment sets a PIN code, generates a key that is recorded in Azure AD, after 30 minutes, Azure AD Connect synchronizes the data and adds a new key to the attribute msDS-KeyCredentialLink user account. As it turned out, the msDS-KeyCredentialLink attribute is an array ... In the course of normal work, for unknown reasons, the user when trying to unlock the computer can not log in (about a month has passed), because the message on the screen "That option is temporarily unavailable". It turns out that the client program generated a new key (the user did not change the PIN code), passed it to Azure AD, and only after the next synchronization of Azure AD Connect, the user will be able to log in! And synchronization can be at least 30 minutes! I.e. the work is paralyzed for up to 30 minutes! And the msDS-KeyCredentialLink attribute in the local AD does not replace the key with a new one, but adds a new one to the array!
Question: why does automatic key generation occur (without the user's knowledge)? why store old keys in the msDS-KeyCredentialLink attribute? How can I avoid downtime if the minimum threshold for Azure AD Connect synchronization can only be set to 30 minutes? When using the Certificate Trust model, the key will not be stored in the msDS-KeyCredentialLink attribute, and this problem can be avoided?
Some details:
PIN-code was established relatively long ago (about a month ago), the computer is running Outlook and Teams, in the morning as usual the user entered PIN-code in the process (today 22.12.2020) it was noticed that Outlook asks to enter an additional password (usually this is not normally a problem with authentication), an attempt was made to lock the computer at 12:15 and discovered this feature described in the question.
but! After 3 hours, history repeats itself! There is no suspicious activity in the Windows security log! Where can I track activity? Is there a problem on the Azure AD Connect side?
More additional information
17:07 the entrance!
The msDS-KeyCredentialLink attribute of the user in local AD contains 4 keys - successful login!
The msDS-KeyCredentialLink attribute contains 3 keys. Access to network resources is not possible, login to the computer is not possible by pin-code.
17:24 appeared 4 the certificate! synchronization took place at 17: 22: 39!
Access to network resources works! I can enter by pin-code.
The msDS-KeyCredentialLink attribute again contains 3 keys ... no one except MSOL_4d60xxx has accessed the attribute! Replica not working correctly between the local domains?!
using-whfbtools-powershell-module-for-cleaning-up-orphaned-windows-hel Got a list of keys from Azure AD, they were all created in November ... it is strange that something started to fail ...
UPDATE
It was the second day of analysis ...
23.12.2020 the second user had a similar problem! look carefully at it!
The user in November created 2 keys (20-11-26 and 20-11-19), in AAD his profile has 2 keys. In AD, the msDS-KeyCredentialLink attribute has an array of 2 keys.
In a domain, for example, 10 domain controllers, the user "LOGONSERVER" is set to #1.
Today, the connector only made the following changes with the user:
When What
23.12.2020 15:41 User 'ivanov.ivan' was modified by 'mydomain\MSOL_xxx' Modified Properties : msDS-KeyCredentialLink, Values : B:854:<Binary>
23.12.2020 15:11 User 'ivanov.ivan' was modified by 'mydomain\MSOL_xxx' Modified Properties : msDS-KeyCredentialLink, Values : B:854:<Binary>
23.12.2020 14:03 User 'ivanov.ivan' was modified by 'mydomain\MSOL_xxx' Modified Properties : msDS-KeyCredentialLink, Values : B:854:<Binary>
23.12.2020 13:33 User 'ivanov.ivan' was modified by 'mydomain\MSOL_xxx' Modified Properties : msDS-KeyCredentialLink, Values : B:854:<Binary>
23.12.2020 10:32 User 'ivanov.ivan' was modified by 'mydomain\MSOL_xxx' Modified Properties : msDS-KeyCredentialLink, Values : B:854:<Binary>,B:856:<Binary>
23.12.2020 10:02 User 'ivanov.ivan' was modified by 'mydomain\MSOL_xxx' Modified Properties : msExchSafeSendersHash
Next, an interesting story turns out!
General state:
AAD, AD: msDS-KeyCredentialLink 2 keys installed (AD on all controllers the same value)
Computer: turned on, logged in, locked
Azure AD Connect: syncing was successful 10 minutes ago
Actions:
1_ The User successfully login using the Windows Hello for Business PIN-code
2_ Instantly, on domain controller #1 "LOGONSERVER" the msDS-KeyCredentialLink attribute becomes a value with 1 key (why???)!
$arrDC = Get-ADDomainController -Filter * -Server mydomain
$arrDC | ForEach-Object -parallel {$value = Get-ADUser -Server $_ -Identity ivanov.ivan -Properties msDS-KeyCredentialLink; write-output "$($value.'msDS-KeyCredentialLink'.count) $($_.HostName) $($value.'msDS-KeyCredentialLink')"}
3_ On local AD controllers, the replica starts and the value is set to 1 key (i.e. there is no correct key) on all controllers. The user begins to have errors with authentication in Outlook and with access to network drives.
4_ If we wait 20 minutes, Azure AD Connect brings msDS-KeyCredentialLink to the correct state (AD replicas update this), but if the user login again, the value will be corrupted again.
My thoughts: as if AD controller #1 "LOGONSERVER" after successful Kerberos authentication (client allows login) decides to delete this key from the user profile!
New Question: why is this happening (something is wrong with the key)? who does this (local controller)? how to predict it or how to get rid of it?
p.s. for the first user, the problem 23.12.2020 itself was eliminated, the attribute is no longer overwritten, the pin-code entry Windows Hello for Business works normally
Hi! We are investigating your issue and will update you shortly!
Best,
James
While this is being investigated, I do have a couple of remarks. It seems you also tried to do on-prem Windows Hello for Business in your environment (since you posted about that few moons ago). If that's the case in this environment as well, and if the user has in fact created a key through ADFS instead of Azure AD and if the user is in the scope of Azure AD Connect synchronization, then what you see would be expected. The key would be owned by AD, then synced to Azure AD Connect's metaverse, then is supposed to go to Azure AD but Azure AD doesn't want it (it has "authority" in a way on Windows Hello for Business for synchronized users) and clears it. Azure AD Connect will show it in the sync logs.
Maybe the challenge to repro comes from that peculiar situation.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 38%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
Same issue here. Works at first auth time but not on subsequent logons to onprem resources like internal websites and drive mappings. And then starts working again after AAD-sync.
Solved it with just patching our 2016 DC.
Also resynced the account that had problems, moved off synced OU in AAD-connect and back again, don't know if that made any difference.
Didnt do the purging
But we also have accounts that can't use pin at all. This started around the time when we removed ADFS from our domain and in the same time upgraded AAD-connect.
Don't know if its related but it feels like it :D
Please start a new thread about this issue and share logs (error message the user gets, when you see in the event logs for WHfB, the output of dsregcmd /status for the impacted users etc...)
@Pierre Audonnet - MSFT , sorry, didn't notice your message in the thread ... yes, we tried to do a local environment with "ADFS" first. But now the infrastructure is only with "Azure AD Connect" and clients create keys in "AAD" and then import them through synchronization in "AD".
I just found one of the known reported issue , I guess this may be related since I dont know unless you have support case and data to review.
Please check the patch on your DCs and validate if that resolve your problem.
@Brian Saville @_KUL
Please validate and comment
=============
Symptom:
Cause:
WHfB public key being removed from local AD user account keycredentiallink attribute directly after the user sign in successfully for the 1st time. This issue was on 2016 Domain Controller due to a missing update.
Resolution:
We made sure that all 2016 domain controllers are up to date including KB4593226. Then, the issue resolved.
===============
hope this helps
Wish you a happy new year
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 47%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBS%3C/text%3E%3C/svg%3E)
@Nagappan Veerappan : these are the exact symptom I am facing. Only difference here is DCs are 2019. I will check on KB4593226 and let you know.
@Brian Saville - I see Dec 8 release patch , one related to 2019 DC is below. Please check this patch
https://support.microsoft.com/en-ie/help/4592440
Hello!
Thank you @Brian Saville for your perseverance! Now we have an idea - update! :)
@Nagappan Veerappan let's start the process of installing the recommended update! Question - why did everything work before? In November and early December, there were no problems! Is this problem somehow related to one-way changes in the Azure AD architecture? Or is it a rollback of changes in the November updates? The reason for the question - now Windows Hello for Bussines is used by a test group of users (all of them have absolutely problems (which I described on the example of one)), this behavior is not acceptable if there is a failure when the technology will work for the entire my enterprise ...
@_KUL The above problem , Support spot that local DC level some regression happens and delete the "msDS-KeyCredentialLink" attribute post first authentication. which causes WHFB unusable. This bug address with those patch.
Once client lost keys, it provision again and add the new key in the multi-value attribute (like Array), writeback happens from AAD to AD via sync cycle after 30 min. Place the new keys. But this problem repeats if you authenticate against problem DC
What I don't know of : What patch caused this behavior on on-prem DCs and when ?
What I know of : This symptoms reported as bug and fixed with patching the DC (with Dec 8 release)
My apologies for any inconvenience caused by this problem and Have a nice day.
Just got update from support one of the case, they have had this problem post update of this patch -September 8, 2020—KB4577015 (OS Build 14393.3930) (microsoft.com)
on 2016DC
However your scenario may defer or different problem if you still have problem post update. Hence feel free to reach out to support with the case. they can validate
Thank you
Nagappan Veerappan
@Nagappan Veerappan , yes, most likely your solution fixed the situation, thank you!
@Nagappan Veerappan thank you very much for your continued active participation in helping and solving our problem!
And thank you to @Pierre Audonnet - MSFT for additional participation!
I wish you success!
unless users run certutil -deleteHelloContainer. we don't delete that "msDs-KeycredentialLink" key. Also as you already aware by now. That msDs-KeycredentialLink attribute is multi valued attribute. This behavior not experienced by other customers. So I bet something specific to that machine or environment causing to loose the keys and re-provisioning back with new keys.
Best choice is open up support case and Support can collect the appropriate logs to investigate further.
Same issue here. Started about the same time. No configuration changes have been made. Very small environment. I am aware of all changes. It just stopped working about 10 days ago. Spent many, many hours investigating to ensure my configuration is correct. Finally came across this thread and am having the exact same issue as reported by @_KUL in his OP. So, @Nagappan Veerappan , this is being experienced by other customers (me!). :)
Any update?
I see, Not sure, if any support case opened for the above issue. if they have one. we can track further
Thanks for reporting as well
I worked with product group and get this listed in known issues. We have this publicly document to benefit future customers.
Hope this helps
-Nagappan Veerappan
Just wondering if anyone had a chance to validate and confirm if that helps @Brian Saville and @_KUL
Happy weekend