Case in The Corner Series: SQL Server Express Edition will not Install

[Today’s post comes to us courtesy of Wayne Gordon McIntyre from Commercial Technical Support]

This case originally started out with the SQL team, since it was an issue with installing SQL Server 2005 Express edition. However, they were trying to install on an SBS 2003 server and the SQL engineer correctly determined that this was an issue with the O/S and not the SQL installer itself. The SQL engineer decided to engage an escalation resource on the SBS team to have a look. The error we were seeing when trying to install SQL was

Since the error referred to the logs for more information we also looked in the SQL Setup log, searched on Return Value 3 (Return Value of 3 in MSI logs indicates a failure). In the logs we saw we were trying to make a connection to the database to run a configuration script but the connection failed. We also know that with SQL Server 2005, connections to the database are made over a secure encrypted connection by default (this is important later). Output from the log is below.

SQL_ERROR (-1) in OdbcConnection::connect sqlstate=08001, level=-1, state=-1, native_error=233, msg=[Microsoft][SQL Native Client]Shared Memory Provider: No process is on the other end of the pipe.

sqlstate=08001, level=-1, state=-1, native_error=233, msg=[Microsoft][SQL Native Client]Client unable to establish connection

Error Code: 0x800700e9 (233)
Windows Error Text: No process is on the other end of the pipe.
Source File Name: lib\odbc_connection.cpp Compiler Timestamp: Fri Jul 29 01:13:53 2005
Function Name: OdbcConnection::connect@connect Source Line Number: 148

---- Context -----------------------------------------------

Connecting to SQL Server
ExecuteSqlCommands
Originial error was 800700e9 (233)
cript
SqlScriptHlpr

Unfortunately this didn’t give us much more information than we already had from the error shown during setup. Next, in our search for information, we went to the event logs, hoping for something more there. We did get an MsiInstaller Event, but as you can see below, it is more of the same.

At this point, I decided to not troubleshoot SQL anymore, and to look for other things that may be failing. Since we know the connection is being attempted over SSL, I tried making SSL connections over the internet to SSL sites. This worked without an issue, I then tried to make an SSL connection to the server itself by going to https://localhost/exchange and it failed to connect. I tried a few other sites hosted on the server and even created my own and made sure it had a good cert and it also failed. So we now know for sure that SSL on the server side is a problem.

At this point I wanted to dig a bit deeper into why SSL connections to the server was failing, for this I used a tool called SSLDiag (publically available tool from Microsoft.com IIS 6 only). When launching SSLDiag it performs an initial diagnostic of your IIS sites that are using SSL. This did not reveal any errors, however you can also right click on one of the sites and simulate the SSL handshake.

After clicking Simulate SSL Handshake the output received as follows.

So now I have an error code to work with. To determine the meaning of this error code, I used "err" (another publically available tool from Microsoft.com).

At this point I decided to compare a working machine (my own server) against his server by taking side by side procmons of a site that is configured with SSL. This is a common technique we use in support to determine how something should work, so we can fill in the missing pieces on the machine that is not working. On the top is the broken server and on the bottom is a working server, the key difference is that on the working server LSASS loads SChanell.dll and the broken server does not (but we also can determine from the procmon that Schanell.dll is fine because on the client piece of the communication thru internet explorer it DOES load schannel.dll)

Now I need to figure out why Schannel is not loading, my first assumption which turned out to be correct is that it is somehow disabled on the server end. So I put my searching skills to use by searching on "SChannel Disabled" in the Microsoft knowledge base and the first hit was:

187498 How to disable PCT 1.0, SSL 2.0, SSL 3.0, or TLS 1.0 in Internet Information Services

https://support.microsoft.com/kb/187498/sl

And sure enough under the key HKLM\System\CurrentControlSet\Control\SecurityProviders\SCHannel\Protocols\SSL 3.0\Server a DWORD of Enabled existed and was set to 0 (this also existed under the TLS 1.0 Protocol Key). I deleted the Enabled DWORD, rebooted the server and I can now access SSL websites hosted on the server. However, the initial reason for the case was still for the SQL installation, since we knew at the beginning it was failing to make a connection to the SQL Server instance over a secure connection, we figured that the SQL install should now work. We attempted the SQL installation again and it completed without an error and the issue was resolved.