100's of Azure AD Connect messages in event log after every reboot

Question

We are running in Hybrid mode using Azure AD Connect ver 1.5.29.0 Running on a Windows 2019 Server Version 1809
The service is running fine until I reboot and then my System event log fills up with 100's of messages over the course of 15 - 30 seconds with the following events:
Event: 7031
Description: The Microsoft Azure AD Sync service terminated unexpectedly. It has done this 1 time(s). The following corrective action will be taken in 0 milliseconds: Restart the service.
Event: 7000
Description: The Microsoft Azure AD Sync service failed to start due to the following error:
The service did not start due to a logon failure.
Event: 7038
Description: The ADSync service was unable to log on as SCIINC\AAD_0315e1ea987c with the currently configured password due to the following error:
The user name or password is incorrect.
I know the username and password work because when i restart the service it stops and restarts with no issues.
I have changed the services to Automatic (Delayed Start) - Same issue.
I have removed and reinstalled Azure AD Connect and still have the same issue.
We monitor these services because I want to know if there is an issue so I get 100s of messages each time the server is rebooted.
Any assistance would be greatly appreciated.

Answer 1

@Paul Hackett

• How're you rebooting these servers? via auto-shutdown, policy, manually, etc..?
• How often are you rebooting this server?
• Is there a specific reason why you're rebooting the server? Maintenance, performance, best practice, etc..?

Based off the error messages you're receiving it looks like it's mainly directed to the service. However, rebooting the actual server is the cause of these service start/stop events along with the AzureAD Sync logon error. I was able to look into the specific events regarding the Service Control Manager, which should hopefully help.

Event7038: ADSync specific doc

Event7000: Service Control Manager - Start operations

Event7031: Service Control Manager - Stop operations

----------

Please let us know if this reply helped resolve your question. If so, please remember to "mark as answer" so that others in the community facing similar issues can more easily find a solution.

Answer 2

James,
I will reboot the server manually as well as via a script depending on what needs to be done. We patch our DCs weekly via a script which when completed reboots the server.
I did patches this week-end, which 2 were applied, KB4561608 and KB4562562 I did NOT receive the multitude of notifications after the reboot. I tried a manual reboot and again no multiple notifications. I verified that the alerts were not in the logs.
My best guess is that the SSU fixed this issue.

Answer 3

That error usually occurs if you have at some point changed the service account password, which might explain why the credentials still work but are failing later.

The documentation mentions that there are two things that need to be done when you change the service account password.

First, you need to change the password under the Windows Service Control Manager. Until this issue is resolved you will see following errors:

If you try to start the Synchronization Service in Windows Service Control Manager, you receive the error "Windows could not start the Microsoft Azure AD Sync service on Local Computer". Error 1069: The service did not start due to a logon failure."

Under Windows Event Viewer, the system event log contains an error with Event ID 7038 and message “The ADSync service was unable to log on as with the currently configured password due to the following error: The user name or password is incorrect."

Second, under specific conditions, if the password is updated, the Synchronization Service can no longer retrieve the encryption key via DPAPI. Without the encryption key, the Synchronization Service cannot decrypt the passwords required to synchronize to/from on-premises AD and Azure AD. You will see errors such as:

Under Windows Service Control Manager, if you try to start the Synchronization Service and it cannot retrieve the encryption key, it fails with error “Windows could not start the Microsoft Azure AD Sync on Local Computer. For more information, review the System Event log. If this is a non-Microsoft service, contact the service vendor, and refer to service-specific error code -21451857952.”

Under Windows Event Viewer, the application event log contains an error with Event ID 6028 and error message “The server encryption key cannot be accessed.”

To ensure that you do not receive these errors, follow the procedures in Abandoning the ADSync service account encryption key when changing the password.

If you haven't done so already I would try following the troubleshooting guide for this error and try those two steps.

Answer 4

Marilee,

Thank you for the information.
Just to clarify the only time I get the errors in the logs are after a reboot of the server and I only get them for 15 - 30 seconds. I don't get them if I restart the service or at any other time. The service has always started when I restart it or after 30 seconds after I reboot.
I read the article Abandoning the ADSync service account encryption key and it mentioned that "The following procedures only apply to Azure AD Connect build 1.1.443.0 or older."
We are running 1.5.29 so this doesn't apply to our situation.
When I was reinstalling the AD Connect it didn't ask me for an account to setup the service with, it just created a new user in our AD, username AAD_0315e1ea987c and set the password.
I followed this article to set it up https://learn.microsoft.com/en-us/azure/active-directory/hybrid/how-to-connect-install-custom
I tried to click on the link you sent for the Troubleshooting guide but I get a page not found. If you can please send me that link I will work through those steps.

Thanks,
Paul