question

AaronKennedy-2673 avatar image
0 Votes"
AaronKennedy-2673 asked ricardosolisvillegas-4678 commented

Azure AD Connect directory sync fails with "SSL/TLS secure channel" error in event log

I have an Azure AD Connect (v2.1.16.0) installation that has recently begun failing during the majority (but not all) of its directory sync attempts. When a directory synchronization fails, the synchronization service manager indicates that it is the import and export tasks using the AAD connector that fail. Here are example errors errors that show up in the event log:

ERROR #1 - ADSync - Event ID: 6005
"The management agent "[tenantID].onmicrosoft.com - AAD" failed on run profile "Delta Import" because of an unspecified management agent error."

ERROR #2 - Directory Synchronization - Event ID: 906
"Authenticate-MSAL: unexpected exception [Unspecified-Authentication-Failure] - extendedMessage: An error occurred while sending the request. | The request was aborted: Could not create SSL/TLS secure channel.
webException: The request was aborted: Could not create SSL/TLS secure channel.
STS endpoint: HTTPS://LOGIN.MICROSOFTONLINE.COM/
[tenantID].ONMICROSOFT.COM"

ERROR #3 - Directory Synchronization - Event ID: 906
"GetSecurityToken: unable to retrieve a security token for the provisioning web service (AWS). An error occurred while sending the request. | The request was aborted: Could not create SSL/TLS secure channel.. extendedMessage: An error occurred while sending the request. | The request was aborted: Could not create SSL/TLS secure channel.
webException: The request was aborted: Could not create SSL/TLS secure channel.
STS endpoint: HTTPS://LOGIN.MICROSOFTONLINE.COM/
[tenantID].ONMICROSOFT.COM"

ERROR #4 - Directory Synchronization - Event ID: 106
"Failed to connect to Windows Azure Active Directory during import: Exception: System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The request was aborted: Could not create SSL/TLS secure channel.
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
"

When I open the Azure AD Connect installation wizard (AzureADConnect.exe), I get a very similar error message when attempting to sign in with a cloud-only global administrator account. However, if I use the /InteractiveAuth command line then I am able to sign in and use the wizard normally (but the post wizard synchronization will still fail).

I have spent three days searching the internet and trying different solutions. I have:
1. Uninstalled and reinstalled Azure AD Connect
2. Installed Azure AD Connect on a different server (with same result)
3. Made sure that the cloud sync account is not using MFA and is excluded from all conditional access policies
4. Made sure that the cloud-only global administrator account does not use MFA (and does use @[tenantID].onmicrosoft.com instead of one of my custom domains)
5. Ensured that all accounts have unchanged and unexpired passwords
6. Updated all of the root CA certificates on my server
7. Enforced TLS 1.2 for all applications (including .NET Framework) on my server
8. Ensured that all the latest .NET updates have been applied
9. I should also point out that my network does NOT use a web proxy

At this point I am stuck and have no idea where my next troubleshooting steps should be. I am suspecting that the heart of the error has something to do with the .NET Framework, but I am completely unfamiliar with .NET and can only understand enough to recognize that Error #4 above is being thrown by .NET. (Sorry if adding the #dotnet-ad tag ends up being irrelevant)

Any help or direction that you can provide would be greatly appreciated.

azure-ad-connectazure-ad-msaldotnet-ad
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

ricardosolisvillegas-4678 avatar image
0 Votes"
ricardosolisvillegas-4678 answered ricardosolisvillegas-4678 commented

Hello @AaronKennedy-2673

Thank you for your heads up.

I have to say that you really did try almost everything to get this issue solved so, great info as well as the steps given.

For now, I have read this other steps that perhaps you might try as well.

https://community.developer.visa.com/t5/Two-way-SSL-X-Pay-Token/NET-Core-3-1-The-credentials-supplied-to-the-package-were-not/m-p/14487/highlight/true#M1165

Let me know if that help you and if not, we will keep assist you on this one.

Looking forward to your feedback,

Cheers,

Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks for the suggestion.

Unfortunately this is not a situation where I am using OpenSSL, so the solution provided in your link will not work for me. When I follow the chain of links on the VISA developer forum, I eventually get to a page describing setting up two-way SSL communication. I have actually already completed those requirements (using the DigiCert global root CA and creating a CA signed certificate for my server).

But I don't want to mess around much with that for the simple fact that my Azure AD Connect directory sync actually does work occasionally. That is the part of this whole scenario that bothers me the most. About 80% of the directory sync attempts fail, but about 20% of the time, the directory sync is successful, and I don't know what causes some to fail and others to succeed.

To make matters worse, this problem began to happen out of the blue with absolutely no changes having occurred on my server or on my network. All of my certificates were updated in May 2022, and my server sits behind WSUS, so no Windows or .NET updates were applied. My AD Connect was working on August 17th, but stopped working (most of the time) on August 18th.

As far as I can tell, there was no change on my end that would have let to this behavior. I just don't get it.


0 Votes 0 ·

Many thanks for your clarification as well as the extra details about it.

Let me double review the whole issue what the next accurate step can be on this and get back to you shortly.

Cheers

0 Votes 0 ·
ricardosolisvillegas-4678 avatar image
0 Votes"
ricardosolisvillegas-4678 answered

Hello @AaronKennedy-2673

I was wondering about what the accurate next step shall be and well, I ended up thinking of this:

  • You could review your Curl version running on your servers and this can be double-check like this:

    C:\Windows\system32>curl --version
    curl 7.79.1 (Windows) libcurl/7.79.1 Schannel
    Release-Date: 2021-09-22
    Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
    Features: AsynchDNS HSTS IPv6 Kerberos Largefile NTLM SPNEGO SSL SSPI UnixSockets

    C:\Windows\system32>

If this is not the latest, please update it...

  • Furthermore, update the PowerShell version running as well.

  • Then, any duplicate attribute can affect this process.

Finally, I wonder what WS server are the ones affected like if they are 2019 or any other.

Looking forward to your feedback,

Cheers,

Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

AaronKennedy-2673 avatar image
0 Votes"
AaronKennedy-2673 answered ricardosolisvillegas-4678 commented

Well, after almost 40 hours of troubleshooting I discovered the solution by accident. I was doing a packet capture on the AD Connect server while forcing it to do an directory sync when it actually succeeded. That was wonderful! I immediately did another forced directory sync which of course failed. But this meant I could examine the two traces to see what the difference was between them.

The difference was DNS. My on-premises DNS server is set up to forward external DNS queries to my ISP, but for redundancy I also have Google public DNS as a failover. The priority is like this:
ISP.DNS.#1
ISP.DNS.#2
8.8.8.8
8.8.4.4
If a directory sync request got forwarded to one of my ISP's DNS servers, the TLS connection would fail. However, on those rare occasions when the directory sync connection got forwarded to Google public DNS, it would succeed.

I have contacted my ISP to let them know what is happening, and in the meantime I have removed the ISP DNS servers from my forwarding table. Ever since making that change I have had 100% successful directory synchronizations. And I don't even need to use the /InteractiveAuth command line switch to access AzureADConnect.exe anymore.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @AaronKennedy-2673

Many thanks for the update as well as the findings about it.

So, basically when that DNS behavior happens as you stated... It is like asymmetric routing issue at some point when this sync process starts.

Well done!

Cheers.

0 Votes 0 ·