Active Directory Issues

Question

Current Domain Environment:
2 Domain Controllers running Server 2016 Standard. Both machines are hosted in VMWare on premise. Both machines have Active Directory Domain Services and DNS Server installed. One server is also running DHCP Server.
1 member server running Server 2019 Datacenter hosting Active Directory Certificate Services and Network Policy Server using Radius as part of MFA for VPN users.
GPO pushes certificates to computers in VPN group
1 member server running Server 2019 Datacenter hosting Microsoft Exchange 2019.
Several Server 2016 and Server 2019 servers hosting various applications and services.
Domain Functional Level 2016

If either of the 2016 Domain Controller servers is restarted for ANY reason, it is usually not possible to log in to the server and Domain services and DHCP do not function. When the password for the user account is entered and the “submit” button or enter key is pressed, nothing happens. The cursor is either sent to the beginning of the password field or to the beginning of the username field. Changing users or retrying the logon does not change the behavior. The machine must be restarted several times before finally being able to logon.
We tried adding 2 new domain controllers using Server 2019 Datacenter. Once it they were promoted to a domain controller, the console could not be logged onto. It would give the message or “Bad Username or Password”. If they was left running for a period of time and a computer on the network tried to authenticate a user account against this domain controller, the account would be denied with “Bad Username or Password”. The server could be connected to and managed with Server Manager or PowerShell. Remote Desktop sessions could not connect and were give the “Bad Username or Password” message. Server 2019 Domain controller was powered off to prevent valid user from being denied.

Running the AD Replication Status Tool, one of the 2019 DCs gives 2 errors for 2 of the other DCs. Replication Error 1256 and 1722. Running DCDIAG from one of the 2016 DCs, all tests pass except it shows "A recent replication atempt failed:" and it lists the 2 errors: 1256 & 1722

Obviously, something is broken and I have exhausted myself trying to track this down.

Answer 1

For clarification:

Local machine IP for first DNS?
Another DC for 2nd?

Answer 2

Things went to complete crap while trying to do all of this. I wound up removing DC01 and DC02 and dong Metadata cleanup. If I reboot either of the 2 remaining DCs I still have to reboot multiple times before being able to log in. Something is still not right.

I re-ran all of the things you asked me to run and re-uploaded the new results to OneDrive.

https://1drv.ms/u/s!AqUWjGdph56LgQu267clVCyFl_SB?e=UmwZtc

Answer 3

Replication is still broken
"no more endpoints available from the endpoint mapper" happens when a service continues to try until all the dynamic ports have been consumed. Rebooting may help.

Also it is difficult to know if the event log errors are pre or post last boot so I usually clear out the relevant ones before rebooting. This way the bogus ones are not picked up by dcdiag

If the DFS Replication event log error's persist you could try a non authoritative sync
https://support.microsoft.com/en-us/help/2218556/how-to-force-an-authoritative-and-non-authoritative-synchronization-fo

--please don't forget to upvote and Accept as answer if the reply is helpful--

Answer 4

I rebooted DC1 but not AD02 and re-ran the commands and uploaded the files to OneDrive again.

One thing that I see from following some of the links is that there is nothing in the Registry under HKLM\SYSTEM\CurrentControlSet\Services\DFSR\Paramaters\SysVols\Seeding SysVols

Answer 5

AD02 DFS Replication service failed to start an RPC listener. Replication is disabled on this machine. This event can be caused by low memory issues

• May need to reboot to recover from this

Then you could try a non authoritative sync from DC1
https://support.microsoft.com/en-us/help/2218556/how-to-force-an-authoritative-and-non-authoritative-synchronization-fo