Failover Cluster Troubleshooting
This topic provides information about the following issues:
Basic troubleshooting steps.
Recovering from a failover cluster failure.
Resolving the most common failover clustering problems.
Using extended stored procedures and COM objects.
Basic Troubleshooting Steps
When working with SQL Server failover clustering, remember that the server cluster consists of a failover cluster instance that runs under Microsoft Cluster Services (MSCS). The instance of SQL Server might be hosted by Microsoft MSCS-based nodes that provide the Microsoft Server Cluster.
If problems exist on the nodes that host the server cluster, those problems may manifest themselves as issues with your failover cluster instance. To investigate and resolve these issues, troubleshoot a SQL Server failover cluster in the following order:
Hardware: Review Microsoft Windows system event logs.
Operating system: Review Windows system and application event logs.
Network: Review Windows system and application event logs. Verify the current configuration against the Knowledge Base article, Recommended Private "Heartbeat" Configuration on a Cluster Server.
Security: Review Windows application and security event logs.
MSCS: Review Windows system, application event, and cluster logs.
SQL Server: Troubleshoot as normal after the hardware, operating system, network, security, and MSCS foundations are verified to be problem-free.
Recovering from Failover Cluster Failure
Usually, failover cluster failure is to the result of one of two causes:
Hardware failure in one node of a two-node cluster. This hardware failure could be caused by a failure in the SCSI card or in the operating system.
To recover from this failure, remove the failed node from the failover cluster using the SQL Server Setup program, address the hardware failure with the computer offline, bring the machine back up, and then add the repaired node back to the failover cluster instance.
For more information, see How to: Create a New SQL Server Failover Cluster (Setup) and How to: Recover from Failover Cluster Failure in Scenario 1.
Operating system failure. In this case, the node is offline, but is not irretrievably broken.
To recover from an operating system failure, recover the node and test failover. If the SQL Server instance does not fail over properly, you must use the SQL Server Setup program to remove SQL Server from the failover cluster, make necessary repairs, bring the computer back up, and then add the repaired node back to the failover cluster instance.
Recovering from operating system failure this way can take time. If the operating system failure can be recovered easily, avoid using this technique.
For more information, see How to: Create a New SQL Server Failover Cluster (Setup) and How to: Recover from Failover Cluster Failure in Scenario 2.
Resolving Common Problems
The following list describes common usage issues and explains how to resolve them.
Problem: Incorrect use of command-prompt syntax to install SQL Server
Issue 1: It is difficult to diagnose Setup issues when using the /qn switch from the command prompt, as the /qn switch suppresses all Setup dialog boxes and error messages. If the /qn switch is specified, all Setup messages, including error messages, are written to Setup log files. For more information about log files, see How to: View and Read SQL Server Setup Log Files.
Resolution 1: Use the /qb switch instead of the /qn switch. If you use the /qb switch, the basic UI in each step will be displayed, including error messages.
Problem: SQL Server cannot log on to the network after it migrates to another node
Issue 1: SQL Server service accounts are unable to contact a domain controller.
Resolution 1: Check your event logs for signs of networking issues such as adapter failures or DNS problems. Verify that you can ping your domain controller.
Issue 2: SQL Server service account passwords are not identical on all cluster nodes, or the node does not restart a SQL Server service that has migrated from a failed node.
Resolution 2: Change the SQL Server service account passwords using SQL Server Configuration Manager. If you do not, and you change the SQL Server service account passwords on one node, you must also change the passwords on all other nodes. SQL Server Configuration Manager does this automatically.
Problem: SQL Server cannot access the cluster disks
Issue 1: Firmware or drivers are not updated on all nodes.
Resolution 1: Verify that all nodes are using correct firmware versions and same driver versions.
Issue 2: A node cannot recover cluster disks that have migrated from a failed node on a shared cluster disk with a different drive letter.
Resolution 2: Disk drive letters for the cluster disks must be the same on both servers. If they are not, review your original installation of the operating system and Microsoft Cluster Service (MSCS).
Problem: Failure of a SQL Server service causes failover
Resolution: To prevent the failure of specific services from causing the SQL Server group to fail over, configure those services using Cluster Administrator in Windows, as follows:
- Clear the Affect the Group check box on the Advanced tab of the Full Text Properties dialog box. However, if SQL Server causes a failover, the full-text search service restarts.
Problem: SQL Server does not start automatically
Resolution: Use Cluster Administrator in MSCS to automatically start a failover cluster. The SQL Server service should be set to start manually; the Cluster Administrator should be configured in MSCS to start the SQL Server service. For more information, see Managing Services.
Problem: The Network Name is offline and you cannot connect to SQL Server using TCP/IP
Issue 1: DNS is failing with cluster resource set to require DNS.
Resolution 1: Correct the DNS problems.
Issue 2: A duplicate name is on the network.
Resolution 2: Use NBTSTAT to find the duplicate name and then correct the issue.
Issue 3: SQL Server is not connecting using Named Pipes.
Resolution 3: To connect using Named Pipes, create an alias using the SQL Server Configuration Manager to connect to the appropriate computer. For example, if you have a cluster with two nodes (Node A and Node B), and a failover cluster instance (Virtsql) with a default instance, you can connect to the server that has the Network Name resource offline using the following steps:
Determine on which node the group containing the instance of SQL Server is running by using the Cluster Administrator. For this example, it is Node A.
Start the SQL Server service on that computer using net start. For more information about using net start, see Starting SQL Server Manually.
Start the SQL Server SQL Server Configuration Manager on Node A. View the pipe name on which the server is listening. It should be similar to \\.\$$\VIRTSQL\pipe\sql\query.
On the client computer, start the SQL Server Configuration Manager.
Create an alias SQLTEST1 to connect through Named Pipes to this pipe name. To do this, enter Node A as the server name and edit the pipe name to be \\.\pipe\$$\VIRTSQL\sql\query.
Connect to this instance using the alias SQLTEST1 as the server name.
Problem: SQL Server Setup fails on a cluster with error 11001
Issue: An orphan registry key in [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.X\Cluster]
Resolution: Make sure the MSSQL.X registry hive is not currently in use, and then delete the cluster key.
Problem: Cluster Setup Error: "The installer has insufficient privileges to access this directory: <drive>\Microsoft SQL Server. The installation cannot continue. Log on as an administrator or contact your system administrator"
Issue: This error is caused by a SCSI shared drive that is not partitioned properly.
Resolution: Re-create a single partition on the shared disk using the following steps:
Delete the disk resource from the cluster.
Delete all partitions on the disk.
Verify in the disk properties that the disk is a basic disk.
Create one partition on the shared disk, format the disk, and assign a drive letter to the disk.
Add the disk to the cluster using Cluster Administrator (cluadmin).
Run SQL Server Setup.
Problem: Applications fail to enlist SQL Server resources in a distributed transaction
Issue: Because the Microsoft Distributed Transaction Coordinator (MS DTC) is not completely configured in Windows, applications may fail to enlist SQL Server resources in a distributed transaction. This problem can affect linked servers, distributed queries, and remote stored procedures that use distributed transactions. For more information about how to configure MS DTC, see Before Installing Failover Clustering.
Resolution: To prevent such problems, you must fully enable MS DTC services on the servers where SQL Server is installed and MS DTC is configured.
To fully enable MS DTC, use the following steps:
In Control Panel, open Administrative Tools, and then open Computer Management.
In the left pane of Computer Management, expand Services and Applications, and then click Services.
In the right pane of Computer Management, right-click Distributed Transaction Coordinator, and select Properties.
In the Distributed Transaction Coordinator window, click the General tab, and then click Stop to stop the service.
In the Distributed Transaction Coordinator window, click the Logon tab, and set the logon account NT AUTHORITY\NetworkService.
Click Apply and OK to close the Distributed Transaction Coordinator window. Close the Computer Management window. Close the Administrative Tools window.
Using Extended Stored Procedures and COM Objects
When you use extended stored procedures with a failover clustering configuration, all extended stored procedures must be installed on a SQL Server-dependent cluster disk. Doing so ensures that when a node fails over, the extended stored procedures can still be used.
If the extended stored procedures use COM components, the administrator must register the COM components on each node of the cluster. The information for loading and executing COM components must be in the registry of the active node in order for the components to be created. Otherwise, the information remains in the registry of the computer on which the COM components were first registered.