Server Unavailability Troubleshooting (Windows Server AppFabric Caching)

  • Article

A Windows Server AppFabric cache cluster consists of several servers, or cache hosts. There are many reasons why a single cache server can be unavailable. In one common scenario, an administrator could decide to reboot one of the cache servers after they apply a recent operating system update. This topic discusses cache server unavailability from the perspective of both an application and the cache cluster.

Diagnosing Server Unavailability

There are several symptoms of server unavailability. First, applications might receive a DataCacheException exception with an error code of RetryLater and an error substatus of CacheServerUnavailable. The following list provides two reasons why an application might get this exception.

  • The cache cluster cannot access the cache configuration store.

  • The application points to one or more cache hosts that are incorrect, unavailable, or down.

  • The cache cluster is down.

  • The application runs as a user who has not been granted access to the cache cluster.

First, verify that you can access the cache configuration store from one of the cache host servers. A simple test is to run the Use-CacheCluster Windows PowerShell command on one of the cache host servers. If the cache configuration store is inaccessible, you receive the following error:

Use-CacheCluster : ErrorCode<ERRPS002>:SubStatus<ES0001>:Invalid provider and connection string read. Please provide the values manually.

If the cache configuration store can be accessed, use the Get-CacheHost command to examine the status of the cache hosts in the cache cluster.

Get-CacheHost

Consider the following example output from the Get-CacheHost command.

HostName : CachePort      Service Name            Service Status Version Info
--------------------      ------------            -------------- ------------
CacheServer1:22233          AppFabricCachingService UP             1 [1,1][1,1]
CacheServer2:22233          AppFabricCachingService UNKNOWN        0 [0,0][0,0]
CacheServer3:22233          AppFabricCachingService UP             1 [1,1][1,1]

The CacheServer2 cache host has an UNKNOWN status. In addition, running the Get-CacheHost command generates an exception in the Windows PowerShell session. This indicates that the CacheServer2 server is unreachable. If an application references CacheServer2 only, the application will not be able to access the cache cluster.

In addition to the example here, if one or more of the cache hosts have a status of DOWN, application can receive similar errors.

If all of the cache hosts are operational, then you should check whether the applications accessing the cache cluster have been given appropriate permissions. In the Event Viewer, search the Admin log for an event with an Event ID of 106. This event states that the "server channel security authorization failed for client {DOMAINNAME\username}".

Resolving Server Unavailability

The following table lists the potential causes of server unavailability and the recommended solution.

Server Unavailability Cause Resolution

The cache configuration store is inaccessible.

Verify that the server that contains the cache configuration store is running. If the XML provider is used, verify the permissions on the file share. If the System.Data.SqlClient provider is used, verify the permissions on the SQL Server and confirm that the SQL Server service is running. For more information, see Configuration Model.

The cache cluster is down (all cache hosts have a status of DOWN).

Start the cache cluster with the Start-CacheCluster command.

One or more cache hosts are down in a running cache cluster.

Start the cache hosts with the Start-CacheHost command.

One or more cache hosts have an UNKNOWN status.

Verify that these cache servers are running and that they can be pinged from the other cache hosts in the cache cluster.

The users who are accessing the cache cluster do not have required permissions.

Use the Grant-CacheAllowedClientAccount to grant the user access to the cache cluster. For more information, see Managing Security (Windows Server AppFabric Caching).

For more guidance on the application exception with an error of Retry and an error substatus of CacheServerUnavailable, see RetryLater (CacheServerUnavailable) (Windows Server AppFabric Caching).

See Also

Concepts

Troubleshooting Server Issues (Windows Server AppFabric Caching)