Troubleshooting Extended Protection (Reporting Services)

Use this topic to troubleshoot problems you experience when using extended protection with SQL Server 2008 R2 Reporting Services. This topic can also be useful when troubleshooting general authentication issues as the root cause of those issues may be the configuration of Extended Protection. For information on recent developments with Extended Protection, see updated information with Extended Protection.

Common issues and support scenarios

Error messages

Error states

Common issues and support scenarios

How do I check the current Extended Protection Settings?

library!DefaultDomain!698!12/29/2009-10:14:49:: i INFO: Initializing RSWindowsExtendedProtectionLevel to 'Off' as specified in Configuration file.

library!DefaultDomain!698!12/29/2009-10:14:49:: i INFO: Initializing RSWindowsExtendedProtectionScenario to 'Proxy' as specified in Configuration file.

How do I know that Extended Protection caused Authentication to fail?

The report server's service trace log file will contain entries similar to the following:

http!rshost!ec0!12/29/2009-11:01:37:: i INFO: Authentication failed with error state: 46

Find the error state number and additional information in the list of error states at end of this topic.

How can I verify if extended Protection checks are being performed?

Enable verbose logging by updating the configuration file reportingservicesservice.exe.config. In the <RStrace> section set:

<add name=”Components” value=”all:4” /> 
  • Restart the service.

  • With verbose logging enabled, authentication will write lines similar to the following, indicating verification of Channel Binding, Service Binding, or both are being performed:

http!rshost!ec0!12/29/2009-11:01:37:: v VERBOSE: Performing Channel Binding Check as Proxy.

http!rshost!7b0!12/29/2009-11:26:23:: v VERBOSE: Performing Service Binding Check.

For more information, see ReportingServicesService Configuration File.

Report Server Catalog

Microsoft SQL Client has not been updated to support extended protection at the time of the SQL Server 2008 R2 release. SQL Client is used to connect to SQL Server data sources and to the Reporting Services catalog database. This limitation in SQL Client impacts Reporting Services in the following ways:

  • The SQL Server that runs the Reporting Services catalog database cannot have extended protection enabled or the report server will fail to connect to the catalog database and return authentication errors.

  • All SQL Servers that are used as Reporting Services report data sources cannot have extended protection enabled or attempts by the report server to connect to the report data source will fail and return authentication errors. A possible work around is to change the Reporting Services data sources to use native providers, and not SQL Client. For example, configure the data sources for the ODBC driver and then the SQL Native Client will be used because it supports Extended Protection.

What happens when I enable Extended Protection but do not configure SSL?

The behavior depends on the setting of RSWindowsExtendedProtectionScenario in the rsreportserver.config configuration file.

If RSWindowsExtendedProtectionScenario is set to Direct and SSL is missing

When RSWindowsExtendedProtectionScenario is Direct and there is no SSL URL reservation for Report Server or Report Manager, the typical experience for users viewing Report Server or Report Manager will be to see an empty Report Manager window.

This is because when SSL is missing and RSWindowsExtendedProtectionScenario is set to Direct, the report server disables the authentication types <RSWindowsNTLM />, <RSWindowsNegotiate />, and <RSWindowsKerberos />. Therefore there are no authentication types enabled, and the report server will not respond to requests. This results in the empty windows.

The trace log file will contain error messages similar to the following:

configmanager!DefaultDomain!938!12/29/2009-11:39:39:: e ERROR: SSL is required on Report Server connections when ExtendedProtectionScenario is set to Direct

configmanager!DefaultDomain!938!12/29/2009-11:39:39:: e ERROR: SSL is required on Report Manager connections when ExtendedProtectionScenario is set to Direct

If RSWindowsExtendedProtectionScenario is set to Proxy and SSL is missing

When RSWindowsExtendedProtectionScenario is set to Proxy and SSL is not part of the environment , the typical experience for users viewing report server or report manager will be to see a credentials prompt page, which will never successfully authenticate. If SSL is configured on the Report Server, adding the 's' to the URL used in the browser for report server or report manager, will solve the problem. For example, https://<uri> will solve the problem of the authentication dialog appearing, if it was due to RSWindowsExtendedProtectionSscenario set to proxy.

The report server's trace log file will include messages similar to:

http!rshost!ec0!12/29/2009-11:01:37:: i INFO: Performing Channel Binding Check as Proxy

http!rshost!76c!12/29/2009-10:26:12:: i INFO: Authentication failed with error state: 45

http!rshost!ec0!12/29/2009-11:01:37:: i INFO: Performing Channel Binding Check as Proxy

http!rshost!ec0!12/29/2009-11:01:37:: i INFO: Authentication failed with error state: 46

After enabling Extended Protection, I can connect in Internet Explorer, but I cannot connect using Report Builder, Management Studio, or any .NET client

This scenario only applies to operating systems prior to Windows 7 and Windows 2008 R2. Prior versions of Windows did not initially release with Extended Protection functionality and, therefore, a computer with the older version of the operating system may not have all of the extended protection updates, including the extended protection update for the .NET framework.

When RSWindowsExtendedProtectionLevel is Allow or Require, a client application running on an operating system that supports extended protection must provide channel binding and sometimes service binding. In this example:

  • Internet Explorer was updated for extended protection as part of the Windows extended protection update.

  • However the .NET Framework has not been patched. The .NET framework is required for .NET clients such as Report Builder and Management Studio.

To diagnose this issue, disable Extended Protection and verify the .NET client application can connect. To disable Extended Protection:

  1. Set RSWindowsExtendedProtectionLevel to Off in the RSReportServer.config file.

  2. Restart the Report Server Service.

The solution is to download all the .NET Framework updates.

Why do Local connections pass authentication when Extended Protection is enabled, but remote connections fail?

When doing a loopback authentication, the operating system skips the authentication mechanism and does not enforce Channel Binding.

Therefore when using an HTTP connection and the configuration settings of:

  • RSWindowsExtendedProtectionLevel set to Require

    AND

  • RSWindowsExtendedProtectionScenario set to Proxy

The local connection will succeed but remote connections will fail.

Depending on the URL used to make the local connection and on the URL Reservations configured on the report server, it is still possible that the local connection will fail with a service binding failure because the SPN that was created from the URL reservation may not match an SPN in the valid SPN list.

The remote connection will always fail if you are making an HTTP connection and there is no gateway sitting in between the client and the report server that is configured to make an SSL connection.

401 Unauthorized error when using a Host Header

This scenario is specific to a scale out deployment of Reporting Services and occurs if Report Manager and Report Server are on same computer and both use host headers and use Windows authentication. For example, if you try to access Report Manager at http://<thename>/reports, you would not see a list reports and folders as expected but would receive an error message of "HTTP 401 (Unauthorized)".

The reference <thename> is not the physical name of the computer and is considered a "Host Header". It is an alternate name for the computer on which Reporting Services is installed. Host Headers are also used as one source for valid SPNs when Reporting Services calculates the valid list of SPNs for Extended Protection.

To prevent the 401 unauthorized error, you will need to add the NetBIOS and Fully Qualified Domain Name (FQDN) for <thename> to the list of BackConnectionHostNames stored in the Windows Registry. Reboot the computer after making the registry change.

For more information on how to make the registry change, see the section "Method 2: Specify host names" in Microsoft Knowledge Base article 896861, You receive error 401.1 when you browse a Web site that uses Integrated Authentication and is hosted on IIS 5.1 or a later version.

Error messages

The following list of messages may be seen in the report server's service trace log file.

Error Message

Type

Cause

Troubleshooting Steps

Missing or Invalid ExtendedProtectionScenario setting

Error

  • RSWindowsExtendedProtectionScenario is not in the rsreportserver.config file.

  • Update the configuration file to the default values.

  • The default value for the setting RSWindowsExtendedProtectionLevel is Off.

  • The default value for the setting RSWindowsExtendedProtectionScenario is Proxy.

SSL is required on Report Server connections when ExtendedProtectionScenario is set to Direct

Error

  • RSWindowsExtendedProtectionScenario is set to Direct.

  • The Direct option requires an SSL connection.

  • Obtain an SSL Certificate and configure an SSL URL for report server and report manager. The same needs to be done for SharePoint Central Administration when using Reporting Services in SharePoint integrated mode.

  • Set RSWindowsExtendedProtectionLevel to Off, until an SSL cert is obtained and configured.

  • Set RSWindowsExtendedProtectionScenario to Any, if no SSL certificate will be configured, and some protection is still desired.

HTTP traffic to Report Server will fail NTLM, Kerberos, Negotiate authentication

Warning

  • RSWindowsExtendedProtectionScenario is set to Direct.

  • Direct requires SSL.

  • An HTTP URl reservation is configured. Authentication on this URL reservation will fail.

  • Use HTTPS to communicate to the report server.

  • Remove the HTTP URL reservation using the Reporting Services configuration manager.

SSL is required on Report Manager connections when ExtendedProtectionScenario is set to Direct

Error

  • RSWindowsExtendedProtectionScenario is set to Direct.

  • Direct requires SSL.

  • Report Manager does not have an SSL URl configured.

  • Obtain an SSL Certificate and configure an SSL URL for report server and report manager. The same needs to be done for SharePoint Central Administration when using Reporting Services in SharePoint integrated mode.

  • Set RSWindowsExtendedProtectionLevel to Off until an SSL certificate is obtained and configured.

  • Set RSWindowsExtendedProtectionScenario to Any if no SSL certificate will be configured, and some protection is still desired.

Direct connections to Report Server will fail NTLM, Kerberos, Negotiate Authentication

Warning

  • RSWindowsExtendedProtectionScenario is Proxy; No SSL URl is configured for Report Manager.

  • When configured this way, the user MUST be making a connection over an SSL channel to the PROXY. The connection can be terminated by a Proxy device. If a direct connection is made to the SSRS server without an SSL channel, then the authentication will fail.

  • Verify clients are using HTTPS only.

  • Verify a proxy is configured which has SSL termination configured.

  • Verify all computers, local and remote are configured to use the proxy for connections to the report server and report manager. This may require DNS updates, browser proxy updates, or custom HOSTS file entries for the local computer.

  • Configure an SSL URL on the Reporting Services server for Report Server and Report Manager or SharePoint Central Administration when using Reporting Services in SharePoint integrated mode. All local connections must use the SSL URL. This assumes no proxy has been configured with termination.

  • Set the Report Manager URL explicitly to use the proxy URL with SSL.

HTTP traffic to Report Manager may fail NTLM, Kerberos, Negotiate authentication

Warning

  • ExtendedProtectionScenario is set to Proxy and an SSL URl is configured for Report Manager.

  • When configured this way, users accessing through a Proxy expect to succeed authentication, even if the connection from the proxy to report manager is not over an SSL channel.

  • Local connections to the report manager must use SSL.

  • Verify clients are using HTTPS (SSL channel) only.

  • Verify a proxy is configured which has SSL termination configured.

  • Verify all computers, local and remote are configured to use the proxy for connections to the report server and report manager. This may require DNS updates, browser proxy updates, or custom HOSTS file entries for the local computer.

  • Configure an SSL URL on the Reporting Services server for report server and report manager. All local connections must use the SSL URL. This assumes no proxy has been configured with termination.

  • Set the report manager URL explicitly to use the proxy URL with SSL.

Error states

This section contains names and descriptions for error codes you may see in the report server's service trace log related to Extended Protection. For more information, see Report Server Service Trace Log.

code

Error state name

Description

44

SNIAUTH_ERRST_SSPIHANDSHAKE_CHANNELBINDINGS_NOTSUPPORTED

The operating system does not support Channel Bindings, but the report server is configured to require Extended Protection. Update the operating system or disable Extended Protection.

45

SNIAUTH_ERRST_SSPIHANDSHAKE_CHANNELBINDINGS_EMPTYORWRONG

The Channel Bindings from the client do not match the established Transport Layer Security channel. The service might be under attack or the data provider might need to be upgraded to support Extended Protection. The connection was closed.

46

SNIAUTH_ERRST_SSPIHANDSHAKE_CHANNELBINDINGS_NULLOREMPTYORWRONG

The Channel Bindings from this client are missing or do not match the established Transport Layer Security channel. The service might be under attack, or the data provider or client operating system might need to be upgraded to support Extended Protection. The connection was closed.

48

SNIAUTH_ERRST_SSPIHANDSHAKE_SERVICEBINDINGS_UNSUPPORTED

The operating system does not support Service Bindings, but the server is configured to require Extended Protection. Update the operating system or disable Extended Protection.

49

SNIAUTH_ERRST_SSPIHANDSHAKE_SERVICEBINDINGS_QUERYCONTEXTATTRIBUTES

QueryContextAttributes could not retrieve Service Bindings. The Windows error code indicates the cause of failure.

51

SNIAUTH_ERRST_SSPIHANDSHAKE_SERVICEBINDINGS_NULL

The server Extended Protection level is set to Allowed or Required, and the client did not provide a Service Principal Name (SPN). To connect, this client must support Extended Protection. You might have to install an operating system service pack that allows for Service Binding and Channel Binding.

52

SNIAUTH_ERRST_SSPIHANDSHAKE_SERVICEBINDINGS_EMPTY

The server Extended Protection level is set to Allowed or Required, and the client did not provide a SPN. To connect, this client must support Extended Protection.

53

SNIAUTH_ERRST_SSPIHANDSHAKE_SERVICEBINDINGS_SERVICECLASSMISMATCH

The Service Class element of the received SPN is not valid.

54

SNIAUTH_ERRST_SSPIHANDSHAKE_SERVICEBINDINGS_IPADDRESSMISMATCH

The IP Address element of the received SPN is not valid.

55

SNIAUTH_ERRST_SSPIHANDSHAKE_SERVICEBINDINGS_HOSTNAMEMISMATCH

The Host element of the received SPN is not valid.

56

SNIAUTH_ERRST_SSPIHANDSHAKE_OOM

A memory allocation failed while validating the received SPN.

57

SNIAUTH_ERRST_SSPIHANDSHAKE_SERVICEBINDINGS_WSASTRINGTOADDRESSFAILEDFORIPV6

WSAStringToAddress was unable to convert the IP Address element of the received SPN to an address structure. The Windows error code indicates the cause of failure.