Cluster with failed DC server in migration

Question

Hi all

We are in a bit off pain here - An upgrade of our clusters DC resulted in the DC died due to a human error (the sync. between them was not finished).
Therefore our cluster is now running headless - all VM are running and are OK.
But the cluster service are not working do to an not working DC in this environment.

We have built a new server with DC installed and that are OK.
We have rejoined all other servers to this new DC.
We have jet to rejoin or build a new cluster for all off this?

Story short:
We are running 2 DC's that have the controll over the cluster.
We want to upgrade this DC from 2016 to 2019 and started the process.
In this process the user missed the last sync. before taking one DC down.
Therefore the last DC do not know what to do, when there is no DC(Global DC) any more.

The setup:
There is 3 nodes running in the cluster service with Windows Server 2016.
There was 2 DC controlling this cluster running Windows Server 2016.
The clusters storage are on a HPE MSA 2052 directly attached to the servers.

What to do?
We are open to any ideas and any Q&A that can help us in the right direction.

Answer 1

Hi All

Here is an update.
The human fail included that there was no backup of the old AD, so that idea was not possible.

We when though the configuration and planned the reset of the hole setup.

• Copy all data - just to be sure.
• Reset node, for node.
• Add them to the new clusters AD.
• Build the cluster.

It was a long working day to get all up and running again.
All thing went as it should and we are running with all again.

Thank you all for the input for all off this.

Answer 2

Hi,

Just checking in to see if the information provided was helpful.
Please let us know if you would like further assistance.

Best Regards,
Vicky

Answer 3

Might want to look at restoring the domain controller from recent known good backup.

--please don't forget to Accept as answer if the reply is helpful--

Answer 4

Thank you @Anonymous for this input.

We have found this article and it gives a good overview to this issue.
But there are some thing left alone as we see it for now.

• We can't connect to the cluster service. We are guessing that this is caused by validation fail when the DC is offline.
• All VM's are still running as they should, but a failover is not possibly at the moment when the cluster service is dead.

If we simplify the guide - we have this troughs:

• Force Cluster service to stop on all nodes.
• Join them to the new domain/DC.
• Reboot them.
• Now run a repair of the AD objects.

We can't see through if we destroy all VM's in this operation.
Or the hyper-V is running by it self all along.

Any input are still much appreciated.

Answer 5

Something here may help.
https://learn.microsoft.com/en-us/windows-server/failover-clustering/cluster-domain-migration

--please don't forget to Accept as answer if the reply is helpful--