The Cluster service is shutting down because quorum was lost

Question

I have a 3 node Windows Failover Cluster with an Always on availability group. All nodes are virtual machines VMware.

Two nodes/replicas are on site A:
Primary replica: REP111 (Synchronous commit)
Secondary replica: REP112 (Synchronous commit)

One node/replica is on site B:
Secondary remote replica: REP113 (Used for Disaster Recovery - Asynchronous commit)

There is no shared storage.

I have just had a look and the current host server is REP113

There is no witness

For some reason the Cluster service shutdown, with the following errors:

Failover Cluster Manager:
EventID 1135 - The cluster service is shutting down because quorum was lost. This could be due to loss of network connectivity between some or all nodes in the cluster.......

Event Viewer - EventID 1135:
Cluster node '' was removed from the active failover cluster membership.

The cluster after a 1.5 minutes came back online and automatically failed over to REP112 from REP111.

I have checked with my network provider, but they could not see any issues\outages.

I have checked the Validate Configuration report on the cluster, which provided two warnings:

1. "The HostRecordTTL is 20 minutes and Microsoft recommends 5 minutes"
2. For each node/replica "Network path is a single point of failure. Suggests adding additional networks to the cluster."

None of the cluster nodes were rebooted at the time that the issue occurred.

I'm not sure what caused the issue the failover cluster to stop and then failover. Could you please advise on how find what the problem was and how to prevent it?

Answer 1

I suggest you read this:
https://learn.microsoft.com/en-us/windows-server/failover-clustering/manage-cluster-quorum#general-recommendations-for-quorum-configuration

Is REP113 have a quorum vote?

Answer 2

Hi @AyJay-9084,

Welcome to Microsoft Q&A!
Please refer to this document about WSFC Health Monitoring and Failover and this article for Monitor SQL Server Always On Availability groups.
You can use quorum-based approach to monitoring overall cluster health and maximize node-level fault tolerance. For more information, please read this document: WSFC Quorum Modes and Voting Configuration (SQL Server).
A quorum node set is a majority of the voting nodes and witnesses in the WSFC cluster. The overall health and status of a WSFC cluster is determined by a periodic quorum vote. The presence of a quorum means that the cluster is healthy and able to provide node-level fault tolerance.
The absence of a quorum indicates that the cluster is not healthy. Overall WSFC cluster health must be maintained in order to ensure that healthy secondary nodes are available for primary nodes to fail over to. If the quorum vote fails, the WSFC cluster will be set offline as a precautionary measure. This will also cause all SQL Server instances registered with the cluster to be stopped.

Best regards,
Seeya

---

If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 3

Looking at theses links, it suggested that a 3 node cluster be Node Majority.
Having looked at the cluster status, the quorum status is set as Majority.

Why would cluster be set offline if quorum was set as Majority? Is there a way of finding out the source of the problem?