Unable to Access a Published Apache Server behind ISA Server using SSL
Consider a scenario where you are publishing a third party web server, in this case an Apache Server that uses HTTPS through ISA Server 2006. Randomly the site doesn’t work, clients are unable to access it and when this happens the publishing rule test button shows the result below (error 0x80090326):
Notice that this error talks about a server certificate error, so clearly it is something during the SSL process.
Reviewing the Data
Using ISA Data Packager in repro mode (web proxy / web publishing template) was possible to collect simultaneous netmon traces from both NICs (internal and external). During those traces it was possible to see that the SSL handshake on the external interface using the certificate that was bound to the Web Listener on ISA was working just fine. Reviewing the SSL Handshake on the internal interface, while ISA was negotiating with the published server (Apache) we had a failure. Here it is the moment of the failure:
ISA APACHE TCP TCP:Flags=......S., SrcPort=24433, DstPort=443, PayloadLen=0, Seq=3108278462, Ack=0, Win=65535 ( ) = 65535
APACHE ISA TCP TCP:Flags=...A..S., SrcPort=443, DstPort=24433, PayloadLen=0, Seq=2120534540, Ack=3108278463, Win=5840 ( Scale factor not supported ) = 5840
ISA APACHE TCP TCP: Flags=...A...., SrcPort=24433, DstPort=443, PayloadLen=0, Seq=3108278463, Ack=2120534541, Win=65535 (scale factor 0x0) = 65535
After finishing the TCP Handshake they start the SSL handshake and this is done by ISA sending the SSL Client Hello as shown below:
ISA APACHE TLS TLS:TLS Rec Layer-1 HandShake
TLSSSLData: Transport Layer Security (TLS) Payload Data
- TLS: TLS Rec Layer-1 HandShake
- TlsRecordLayer: TLS Rec Layer-1 HandShake
ContentType: HandShake
- Version: TLS 1.0
Major: 3 (0x3)
Minor: 1 (0x1)
Length: 88 (0x58)
- SSLHandshake: SSL HandShake ClientHello(0x01)
HandShakeType: ClientHello(0x01)
Length: 84 (0x54)
- ClientHello: TLS 1.0
+ Version: TLS 1.0
+ RandomBytes:
SessionIDLength: 16 (0x10)
SessionID: Binary Large Object (16 Bytes)
CipherSuitesLength: 22
+ TLSCipherSuites: TLS_RSA_WITH_RC4_128_MD5 { 0x00,0x04 }
+ TLSCipherSuites: TLS_RSA_WITH_RC4_128_SHA { 0x00,0x05 }
+ TLSCipherSuites: TLS_RSA_WITH_3DES_EDE_CBC_SHA { 0x00,0x0A }
+ TLSCipherSuites: TLS_RSA_WITH_DES_CBC_SHA { 0x00,0x09 }
+ TLSCipherSuites: TLS_NTRU_NSS_WITH_AES_256_CBC_SHA { 0x00, 0x64 }
+ TLSCipherSuites: TLS_NTRU_NSS_WITH_3DES_EDE_CBC_SHA { 0x00, 0x62 }
+ TLSCipherSuites: TLS_RSA_EXPORT_WITH_RC4_40_MD5 { 0x00,0x03 }
+ TLSCipherSuites: TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5 { 0x00,0x06 }
+ TLSCipherSuites: TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA { 0x00,0x13 }
+ TLSCipherSuites: TLS_DHE_DSS_WITH_DES_CBC_SHA { 0x00,0x12 }
+ TLSCipherSuites: TLS_NTRU_NSS_WITH_AES_128_CBC_SHA { 0x00, 0x63 }
CompressionMethodsLength: 1 (0x1)
CompressionMethods: 0 (0x0)
ExtensionsLength: 5 (0x5)
Right after that Apache sends a SSL Encrypt Alert error as shown below:
APACHE ISA TLS TLS:TLS Rec Layer-1 Encrypted Alert
TLSSSLData: Transport Layer Security (TLS) Payload Data
- TLS: TLS Rec Layer-1 Encrypted Alert
- TlsRecordLayer: TLS Rec Layer-1 Encrypted Alert
ContentType: Encrypted Alert
- Version: TLS 1.0
Major: 3 (0x3)
Minor: 1 (0x1)
Length: 2 (0x2)
EncryptedData: Binary Large Object (2 Bytes)
15 03 01 00 02 02 2F
....../
By looking to the last two bytes of the hex value under Encrypted data we can find the meaning of the alert:
o 2F in Hex = 47 in Decimal
o 47 maps to illegal_parameter(47) error according to TLS RFC (https://www.ietf.org/rfc/rfc2246.txt?number=2246)
Note: thanks to sudeepg for this nice approach reading SSL Encrypt Alert.
Apache Server is saying that the TLS SSL Client Hello sent by ISA as an illegal parameter for this SSL negotiation.
Conclusion
This problem can happen because MS10-049 which is installed on ISA Server. As a temp workaround this update was removed and the issue got resolved on this particular scenario. However, ultimately you should not remove this update; you should talk to the third party company web server admin and discuss the CVE-2009-3555 with him and how their product adequate for that. If you are publishing an IIS Server you might have this issue too, if you do, read the post below to see how to fix it:
Comments
- Anonymous
December 03, 2010
Nice 1 Bud