Visual Overview -

Technical Overview

SSMS.exe here successfully calls the EndPointMapper on SQL1.Fabrikam.com and is returned the network address SSMS.exe may call back on to work with the DCOM server.

This is the successful call to RemoteCreateInstance, after 10 seconds, client then aborts. So why?  (shortended)

- ScmReply:

- OxidBindings:

- Bindings:

- StringBindings:

TowerId: 7 (0x7)

NetworkAddress: sql1[49299]

- StringBindings:

TowerId: 7 (0x7)

NetworkAddress: 192.168.1.100[49299]

SSMS.exe will choose the first address in list of network addreses and so ends up trying to call: SQL1. Now because SQL is the name of a local machine, DNS correct returns the IP 192.168.2.200 and so the next outbound connection will be headed to the wrong SQL instance. This connection must fail.

If you’re looking at the network tracing, it may look like an abrupt end, caused by the client resetting the connection after having several successful calls to the correct server. Between those calls will be the new session with DNS and the local SQL machine.

This example can play out with other DCOM applications however, in my experience this is a rare situation.

Currently, there are only two workarounds:

1. Rename one of the machines

2. Use a host file entry to direct the netBIOS name to the address desired

General Lesson

Try not to duplicate server names between your domains; though ideally, the FQDN should be returned from the DCOM.

Further Reading

DCOM Servers are responsible for registering calling RpcEpRegister and supplying Binding details or relying on existing Bindings. Control over the Binding name then is due to the DCOM server implementer.

Research Details:

[MS-DCOM] 2.2.19.3

STRINGBINDING.aNetworkAddr.servername = MUST BE one of the following{ A NETBIOS Name, FQDN, IPv4, IPv6, } *removed other requirements to shorten output for our example.

From <https://msdn.microsoft.com/en-us/library/windows/desktop/aa378456(v=vs.85).aspx> ]]> https://docs.microsoft.com/archive/blogs/tdevere/microsoft-jdbc-driver-4-2-contains-a-version-of-sqljdbc_xa-which-creates-a-handle-leak-in-sql-server Mon, 13 Jun 2016 16:04:57 GMT https://blogs.msdn.microsoft.com/tdevere/?p=155  

Recently I worked with a customer who was experiencing a crash in his SQL Server process. He was under the impression that this same code was working in another location but in this envirnonment, they were stressing the box to determine its capacity. That’s when they discovered the crash which eventually led us to find a handle leak in the process.

Steps To Verify Problem

1. Your application uses XA Transactions

2. The sqljdbc_xa.dll file version is 4.2.6420.100

3. Handle count increases with every call [task manager; performance monitor\process\handle count]

4. Once you have exhausted kernel resources, you may also notice related errors:

Log Name:      Application
Source:        Microsoft-Windows-Complus
Date:          6/8/2016 6:36:45 PM
Event ID:      4689
Task Category: (8)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      SQL
Description:
The run-time environment has detected an inconsistency in its internal state. This indicates a potential instability in the process that could be caused by the custom components running in the COM+ application, the components they make use of, or other factors. Error in COM\complus\src\inc\UTSem.H(100), Insufficient system resources exist to complete the requested service.

: CreateEvent returned a NULL handle.

5. This state of affairs eventually leads to a crash in SQL Server Process

Recommendation:

Revert to 4.0 version of JDBC Driver: https://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=11774

Method 1: x86 With Public Symbols mscorwks

1. List stack objects and find the System.__ComObject
 
!wstack
05cb517c 04a28de4   0  2         16 System.__ComObject //This is our reference to the COM component
 
2. Now let's get the SyncBlock Address -4 and 1 level deep (to keep our output clean and simple)
 
0:037> dw 04a28de4-4 l1
04a28de0  0002
 
3. Now we can get the SyncBlock Address directly:

0:037> !sos.syncblk 0002
Index SyncBlock MonitorHeld Recursion Owning Thread Info  SyncBlock Owner
2 0597007c            0         0 00000000     none    05b7eb90 System.__ComObject

4. I haven't these steps on x64 so I would expect this to fail.
 
0:037> ?0597007c+1c
Evaluate expression: 93782168 = 05970098

0:037> dc 05970098 l1
05970098  0053afa0  //InteropSyncInfo

0:037> ?0053afa0+c
Evaluate expression: 5484460 = 0053afac

0:037> dc 0053afac l1
0053afac  004cdfd0  //RCW
 
0:037> ?0x004cdfd0+0x64
Evaluate expression: 5038132 = 004ce034
 
0:037> dc 004ce034 l1
004ce034  004d9b44  //IUnknown
 
0:037> dc 004d9b44 l1
004d9b44  68f289a0  //Vtable

5. You will need symbols for the COM component to know if this is a valid Vtable:

0:037> dds 68f289a0
68f289a0  68e559f0 example!Com::Component::QueryInterface
68f289a4  68e4ba80 example!Com::Component::AddRef
68f289a8  68e55a20 example!Com::Component::Release

Some follow up notes. If you don't have symbols but you see a component listed in the output, you could unassemble one of the functions to determine if it actually looks like the beginning of a function. Also, you could look the component up in the registry and see if it's a registered COM component. Also, depending upon the symbols you have, you may be able to find this information, regardless of the offset values I've used in the example, by noting the types I've highlighted in yellow in the above example.

So our customer's goal was to update their native C++ application with support for calling a static instance of a C# class and share that instance with native and managed clients from within the same process space. This is a sample created to demonstrate one method to accomplish this task. (Many thanks to Rodney Viana for helping solve several critical problems I ran into making this sample work (if you have not downloaded Rodney's debugging extension - what are you waiting for? https://netext.codeplex.com/)

We chose a mixed C++/CLI adapter as a solution:

Visual C++ supports the use of the Standard C++ Library, the Common RunTime library (CRT), ATL, and MFC for applications compiled with /clr (Common Language Runtime Compilation). This allows existing applications that use these libraries to use .NET Framework features as well. From <https://msdn.microsoft.com/en-us/library/ms173268.aspx>

Solution contains three projects (Link to Visual Studio Solution: https://1drv.ms/1A45F7Q):

1. shared_object - C# Class project (.Net 4.5.1)
2. mixedAdapter - C++ CLR project
3. client - C++ Win32

shared_object

This DLL exposes a simple singleton class with one method named Hello_World. It returns a string to the caller.

mixedAdapter

This is the C++ CLR project which acts as an adapter or wrapper. It can been called from C++ natively and it can add managed references. I have a project reference to shared_object. Make sure to add this to your class __declspec(dllexport) so your client project can import later:

   1: #include <msclr\marshal_cppstd.h>

   2:  

   3: #include <string>

   4:  

   5: using namespace System; 

   6:  

   7: using namespace shared_object;

   8:  

   9: #pragma native

  10:  

  11: namespace mixedAdapter 

  12:  

  13: {

  14:  

  15: class __declspec(dllexport) InTheMix

  16:  

  17: {

  18:  

  19: public:

  20:  

  21: std::string SayHello()

  22:  

  23: {

  24:  

  25: SingletonObj^ instance = SingletonObj::GetInstance();

  26:  

  27: std::string returnValue = msclr::interop::marshal_as<std::string>(instance->Hello_World());

  28:  

  29: return returnValue;

  30:  

  31: }

  32:  

  33: };

  34:  

  35: }

  36:  

client

Client has a header file with __declspec(dllimport) to import the definition from the lib file

   1: #include <string>

   2:  

   3: namespace mixedAdapter

   4:  

   5: {

   6:  

   7: class __declspec(dllimport) InTheMix

   8:  

   9: {

  10:  

  11: public:

  12:  

  13: std::string SayHello();

  14:  

  15: };

  16:  

  17: }

  18:  

And the client.cpp file has #pragma comment (lib,"mixedAdapter") added which is a hint to the linker to find the correct lib file. To get this working, I opened to project properties of client, under Linker, I selected "Additional Library Directories" and put in the path of my mixedAdapter.lib file. Without this step you may get a complier error.

   1: #include "stdafx.h"

   2:  

   3: #include "mixedAdapter.h"

   4:  

   5: using namespace mixedAdapter;

   6:  

   7: #pragma comment (lib,"mixedAdapter")

   8:  

   9: int _tmain(int argc, _TCHAR* argv[])

  10:  

  11: {

  12:  

  13: std::string str;

  14:  

  15: InTheMix myClass;

  16:  

  17: str = myClass.SayHello();

  18:  

  19: printf("%s", str.c_str());

  20:  

  21: return 0;

  22:  

  23: }

  24:  

Test:

C:\Visual Studio 2013\Projects\mixedAdapter\Debug>client.exe

Hello World

My customer is attempting to setup a WCF service behind a SSL balancer. The WCF Service will not have a SSL certificate installed so it will be insecure transport between load balancer and the WCF service. The SSL device requires SSL connection and a client certificate.

Customer wrote a simple client application which could not make even an HTTP GET to the service page. The error was The request was aborted: Could not create SSL/TLS secure channel.

Looking at the Netmon traces I could see the following:

client test.org TLS TLS:TLS Rec Layer-1 HandShake: Client Hello.
test.org client TLS TLS:TLS Rec Layer-1 HandShake: Server Hello.; TLS Rec Layer-2 HandShake: Certificate.
client test.org TLS TLS:TLS Rec Layer-1 HandShake: Certificate. Client Key Exchange.; TLS Rec Layer-2 Cipher Change Spec; TLS Rec Layer-3 HandShake: Encrypted Handshake Message.
test.org 10.53.243.7 TLS TLS:TLS Rec Layer-1 Encrypted Alert // This is a Fatal Error: handshake failure

In our case, the customer was able to make the same request using IE. So I asked for IE traces to compare the differences.

client test.org TLS TLS:TLS Rec Layer-1 HandShake: Client Hello.
test.org client TLS TLS:TLS Rec Layer-1 HandShake: Server Hello.; TLS Rec Layer-2 HandShake: Certificate.
test.org client TLS TLS:Continued Data: 1328 Bytes
client test.org TLS TLS:TLS Rec Layer-1 HandShake: Certificate // THIS WAS MISSING IN THE FAILING TRACE
test.org client TLS TLS:TLS Rec Layer-1 Cipher Change Spec; TLS Rec Layer-2 HandShake: Encrypted Handshake Message.

+ Ethernet
+ Ipv4:
+ Tcp
  TLSSSLData: Transport Layer Security (TLS) Payload Data
- TLS: TLS Rec Layer-1 HandShake: Certificate.
  - TlsRecordLayer: TLS Rec Layer-1 HandShake:
     ContentType: HandShake:
   + Version: TLS 1.0
   - SSLHandshake: SSL HandShake Certificate(0x0B)
      HandShakeType: Certificate(0x0B)
    - Cert: 0x1
       CertLength: 2700 (0xA8C)
     - Certificates:
        CertificateLength: 1392 (0x570)
      + X509Cert: Issuer: Entrust Certification Authority
     - Certificates:
        CertificateLength: 14241015 (0xD94CF7)
      + X509Cert: Issuer: , Subject:

Here the HandShake: Certificate is a client certificate the IE client had cached and was automatically sending to the SSL load balancer. Earlier that day the customer had selected the certificate so during our investigation I never noticed the requirement for client certificate.

We resolved this by modifying the customer's client.app.config from:

   <wsHttpBinding>
    <binding name="wsHttpBinding_IUnit">
     <security mode="Transport">
      <transport clientCredentialType="None" />
      <message clientCredentialType="Certificate" />
     </security>
    </binding>
   </wsHttpBinding>

To:

   <wsHttpBinding>
    <binding name="wsHttpBinding_IUnit">
     <security mode="Transport">
      <transport clientCredentialType="Certificate" />
     </security>
    </binding>
   </wsHttpBinding>

More Details

Here’s the failure from inside the System.Net traces

System.Net.Sockets Verbose: 0 : [4272] 00000005 : 02 28       // This is a Fatal Error: handshake failure
    DateTime=2015-02-27T17:46:24.5449298Z
System.Net.Sockets Verbose: 0 : [4272] Exiting Socket#19612312::Receive()  -> Int32#2
    DateTime=2015-02-27T17:46:24.5459298Z
System.Net Information: 0 : [4272] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 1b2a1360:1e8cf70, targetName = test.org, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
    DateTime=2015-02-27T17:46:24.5469298Z
System.Net Information: 0 : [4272] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=IllegalMessage).
    DateTime=2015-02-27T17:46:24.5479298Z
System.Net.Sockets Verbose: 0 : [4272] Socket#19612312::Dispose()
    DateTime=2015-02-27T17:46:24.6669298Z
System.Net Error: 0 : [4272] Exception in HttpWebRequest#32615199:: - The request was aborted: Could not create SSL/TLS secure channel..
    DateTime=2015-02-27T17:46:24.6679298Z
System.Net Error: 0 : [4272] Exception in HttpWebRequest#32615199::GetResponse - The request was aborted: Could not create SSL/TLS secure channel..
    DateTime=2015-02-27T17:46:24.6689298Z

Enabling ETW tracing across your cluster can be made easier by utilizing this handy guide. Method 1 shows how using PsExec and Method 2 shows how to use native Logman.exe only.

Method 1: Using PsExec From SysInternals

Instructions

1. Install PsExec.exe from: https://technet.microsoft.com/en-us/sysinternals/bb897553.aspx

2. Make sure PsExec.exe is in the system path or simply copy PsExec.exe to the c:\windows\system32 directory.

3. Edit the computer_name_list.txt to add cachehost to the trace data collections processes

4. Enable tracing by executing from the command prompt

for /F %i IN (computer_name_list.txt) DO psexec -accepteula -c appfabric_enable_trace.bat -v \\%i

5. To Start Tracing

for /F %i IN (computer_name_list.txt) DO psexec -accepteula -c appfabric_start_trace.bat -v \\%i

5. To Stop Tracing on all Systems run the following command:

for /F %i IN (computer_name_list.txt) DO psexec -accepteula -c appfabric_stop_trace.bat -v \\%i

6. To Delete the AppFabric Trace Log run this command:

for /F %i IN (computer_name_list.txt) DO psexec -accepteula -c appfabric_delete_trace.bat -v \\%i

7. When tracing is complete, copy each of the AppFabCacheOut.etl from each system and send them to your support professional. You can use these commands to assist in that effort:

for /F %i IN (computer_name_list.txt) DO md %i

for /F %i IN (computer_name_list.txt) DO copy \\%i\c$\temp\AppFabCacheOut*.etl c:\temp\%i

Download source files: Enable_AppFabric_Tracing.zip 

Method 2: Using Logman.exe Only

Any of the commands we ran from the command line above can be configured to point to a remote server, without the use of PsExec.exe. However, you’ll need to add a firewall rule to allow this traffic to the cachehost. First, let me translate a command then I will show you the firewall rule you need to create. If you have a number of machine to perform this on you can save some time creating a machine list as we did in step three above: (Edit the computer_name_list.txt to add cachehost to the trace data collections processes)

Logman example using –s switch

for /F %i IN (computer_name_list.txt) DO logman delete CacheETWTracing -s %I

If you receive the error below you need to setup a firewall rule on the target machines to allow logman.exe to run on that remote machine.

Error:

The RPC server is unavailable.

Create a Rule for the program %SystemRoot%\System32\plasrv.exe tighten down to your requirements from that point.

If you’re having to work with the Microsoft Distributed Transaction Coordinator transaction logs, you must already be having a very fun day. To add to your joy, I’ve recently learned that MSDTC does not correctly log XA transaction details within the Transaction Logs. Let me give you some details that I’ve picked up.

Two Scenarios for xa_commit

With TMONEPHASE

The TMONEPHASE flag notifies us the caller is using one-phase commit optimizations.  If you want to dump out the transaction logs here you will only find the following entry for a commit (formatted to read more clearly):

pid=5888 tid=5024 time=01/23/2015-12:14:30.098 seq=758 eventid=RM_ENLISTED_IN_TRANSACTION

Compare this to the Transaction::Commit call which utilize one-phase commit optimizations (I’ve bolded what is missing):

pid=5524           tid=5404           time=01/23/2015-12:11:48.822       seq=71             TRANSACTION_BEGUN                   
pid=5524           tid=5404           time=01/23/2015-12:11:48.822       seq=72             RM_ENLISTED_IN_TRANSACTION          
pid=5524           tid=5404           time=01/23/2015-12:11:48.822       seq=73             RECEIVED_COMMIT_REQUEST_FROM_BEGINNER
pid=5524           tid=5404           time=01/23/2015-12:11:48.822       seq=74             TRANSACTION_COMMITTED               

Without TMONEPHASE

The situation is better when we do use the one-phase optimizations:

pid=5888           tid=5808           time=01/23/2015-12:14:46.586       seq=3473           eventid=RM_ENLISTED_IN_TRANSACTION
pid=5888           tid=5808           time=01/23/2015-12:14:46.586       seq=3474           eventid=RM_ENLISTED_IN_TRANSACTION
pid=5888           tid=5808           time=01/23/2015-12:14:46.586       seq=3475           eventid=RM_ISSUED_PREPARE         
pid=5888           tid=5808           time=01/23/2015-12:14:46.586       seq=3476           eventid=RM_ISSUED_PREPARE         
pid=5888           tid=5808           time=01/23/2015-12:14:46.601       seq=3477           eventid=RM_VOTED_COMMIT           
pid=5888           tid=5808           time=01/23/2015-12:14:46.601       seq=3478           eventid=RM_VOTED_COMMIT           
pid=5888           tid=5808           time=01/23/2015-12:14:46.617       seq=3479           eventid=RM_ISSUED_COMMIT          
pid=5888           tid=5808           time=01/23/2015-12:14:46.617       seq=3480           eventid=RM_ISSUED_COMMIT          
pid=5888           tid=5808           time=01/23/2015-12:14:46.617       seq=3481           eventid=RM_ACKNOWLEDGED_COMMIT    
pid=5888           tid=5808           time=01/23/2015-12:14:46.617       seq=3482           eventid=RM_ACKNOWLEDGED_COMMIT    

Compare this to the Transaction::Commit call which does not utilize one-phase commit optimizations (I’ve bolded what is missing)::

pid=5524           tid=5380           time=01/23/2015-12:12:03.348       seq=12141          TRANSACTION_BEGUN                    
pid=5524           tid=5980           time=01/23/2015-12:12:03.348       seq=12142          RM_ENLISTED_IN_TRANSACTION           
pid=5524           tid=5380           time=01/23/2015-12:12:03.348       seq=12143          RM_ENLISTED_IN_TRANSACTION           
pid=5524           tid=5380           time=01/23/2015-12:12:03.348       seq=12144          RECEIVED_COMMIT_REQUEST_FROM_BEGINNER
pid=5524           tid=5380           time=01/23/2015-12:12:03.348       seq=12145          RM_ISSUED_PREPARE                    
pid=5524           tid=5380           time=01/23/2015-12:12:03.348       seq=12146          RM_ISSUED_PREPARE                    
pid=5524           tid=5380           time=01/23/2015-12:12:03.348       seq=12147          RM_VOTED_COMMIT                      
pid=5524           tid=5980           time=01/23/2015-12:12:03.348       seq=12148          RM_VOTED_COMMIT                      
pid=5524           tid=5980           time=01/23/2015-12:12:03.364       seq=12149          TRANSACTION_COMMITTED                
pid=5524           tid=5980           time=01/23/2015-12:12:03.364       seq=12150          RM_ISSUED_COMMIT                     
pid=5524           tid=5980           time=01/23/2015-12:12:03.364       seq=12151          RM_ISSUED_COMMIT                     
pid=5524           tid=5980           time=01/23/2015-12:12:03.364       seq=12152          RM_ACKNOWLEDGED_COMMIT               
pid=5524           tid=5380           time=01/23/2015-12:12:03.364       seq=12153          RM_ACKNOWLEDGED_COMMIT               

I’ll be following this entry up later with some information about XA performance compared to OleTx transactions.

My customer was migrating from Windows 2003 to Windows 2012 R2. Part of the application still ran classic ASP and VB6 COM+ components in IIS. Their COM+ component requires ASP sessions state so IISIntrinsics was enabled for the component. However, in the new Windows 2012 R2 environment, when the VB6 component requested a value from Session the COM application would fail and the COM+ runtime threw this error:

Server Application ID: {G846F1C5-9A3F-4A78-B513-CDEA1048DB9C}

Server Application Instance ID:

{10232175-0BC5-404A-8440-A8EE01644BAD}

Server Application Name: myCOMPlusServer

Error Code = 0x800a01a8 :

COM+ Services Internals Information:

File: com\complus\src\comsvcs\jit\jit.cpp, Line: 538

Comsvcs.dll file version: ENU 2001.12.10530.16384 shp

So why is the Session not available for the COM+ application even though IISIntrinsics was enabled for the COM+ component?

Debugging Notes

The thread handling the ASP call was marked as MULTITHREADED but the VB6 component was compiled with APARTMENTTHREADED. The effect of this combination forces the thread handling the incoming ASP call to switch threads and allow COM marshalling to call into a different COM apartment. The call to the APARTMENTTHREADED context though, does not pass the IISIntrinsics along which is why it’s unavailable to our COM object despite having IISIntrinsics enabled.

So was this the default behavior? And can we control the apartment type IIS uses to launch ASP applications? The answers are No and Yes. Inside the IIS Service manager, the executeInMta was set to ‘true’ for my customer and by default it should be false.

We took network traces on the failing system and found the Client Hello call received an Encrypted Alert from the remote web service:

- TLS: TLS Rec Layer-1 HandShake: Client Hello.

   - SSLHandshake: SSL HandShake ClientHello(0x01)

      HandShakeType: ClientHello(0x01)

      Length: 111 (0x6F)

    + ClientHello: TLS 1.0

 

- TLS: TLS Rec Layer-1 Encrypted Alert

  - TlsRecordLayer: TLS Rec Layer-1 Encrypted Alert

     ContentType: Encrypted Alert

   + Version: TLS 1.0

     Length: 2 (0x2)

     EncryptedData: Binary Large Object (2 Bytes)

The Encrypted Alert value can be seen with Netmon in the Hex Details:

We can convert the hex data to a meaningful message using the specification for TLS 1.0 from https://www.ietf.org/rfc/rfc2246.txt

7.2. Alert protocol

fatal(2)

handshake_failure(40) (28 to hex is 40)

So the remote server is reporting a handshake failure. So we asked, what is different about this client request and our working clients? The answer was our failing client was missing any RC4 ciphers:

       CipherSuitesLength: 20

     + TLSCipherSuites: TLS_RSA_WITH_AES_128_CBC_SHA         

     + TLSCipherSuites: TLS_RSA_WITH_AES_256_CBC_SHA          

     + TLSCipherSuites: TLS_RSA_WITH_3DES_EDE_CBC_SHA        

     + TLSCipherSuites: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA    

     + TLSCipherSuites: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA   

     + TLSCipherSuites: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA 

     + TLSCipherSuites: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA

     + TLSCipherSuites: TLS_DHE_DSS_WITH_AES_128_CBC_SHA 

     + TLSCipherSuites: TLS_DHE_DSS_WITH_AES_256_CBC_SHA

     + TLSCipherSuites: TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA

The working client request contained these ciphers additionally:

TLS_RSA_WITH_RC4_128_SHA

TLS_RSA_WITH_RC4_128_MD5

And within the ServerHello, we can see the server has preferred the TLS_RSA_WITH_RC4_128_SHA=

- SSLHandshake: SSL HandShake ServerHello(0x02)

       TLSCipherSuite: TLS_RSA_WITH_RC4_128_SHA                { 0x00,0x05 }

So what mechanism controls whether the client sends this RC4 cipher? The answer depends on a number of factors.

Microsoft Security Advisory 2868725

Published: November 12, 2013

https://technet.microsoft.com/library/security/2868725

This update supports the removal of RC4 as an available cipher on affected systems through registry settings. It also allows developers to remove RC4 in individual applications through the use of the SCH_USE_STRONG_CRYPTO flag in the SCHANNEL_CRED structure. These options are not enabled by default.

Update for Disabling RC4 in .NET TLS

Published: May 13, 2014 | Updated: July 8, 2014

https://technet.microsoft.com/library/security/2960358

My customer was at a later version of System.dll and Schannel.dll which contained these security updates and both systems were the same on version. Now, we went to check to see if the ciphers were disabled:

How to completely disable RC4

Clients that deploy this setting will be unable to connect to sites that require RC4, and servers that deploy this setting will be unable to service clients that must use RC4.

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Ciphers\RC4 128/128]

 "Enabled"=dword:00000000

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Ciphers\RC4 40/128]

 "Enabled"=dword:00000000

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Ciphers\RC4 56/128]

 "Enabled"=dword:00000000

However, my customer did not have these values set on the working or failing system.

This caused us to look for another explanation altogether which could explain why .NET clients were failing to use the RC4 cipher. We discovered that in our failing environment the following registry key was enabled:

SchUseStrongCrypto

https://msdn.microsoft.com/en-us/library/windows/desktop/aa379810(v=vs.85).aspx

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v4.0.30319\SchUseStrongCrypto

Instructs Schannel to disable known weak cryptographic algorithms, cipher suites, and SSL/TLS protocol versions that may be otherwise enabled for better interoperability.

After setting this value to 0 and rebooting the system, the client began to send RC4 ciphers.This is very important to note: that you should not use weak ciphers. A detailed response on the topic from Microsoft may be found here https://blogs.technet.com/b/srd/archive/2013/11/12/security-advisory-2868725-recommendation-to-disable-rc4.aspx.

Summary: (updated 3/4/2015)

1. Set your .NET Client to use SSL3 https://msdn.microsoft.com/en-us/library/system.net.servicepointmanager.securityprotocol(v=vs.110).aspx

2. Set the following registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v4.0.30319\SchUseStrongCrypto = 0. This disables the new security update disabling RC4 Ciphers.

3. If the your application is 32bit running on x64 windows, you’ll need to modify the same key under the HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\v4.0.30319

A customer came to me recently asking about options to migrate a set of SOAP services hosted on Windows 2003 into two new modern datacenters running Windows 2012. The most critical requirement for the customer was that the client remain unaware of the change and client proxies could not be modified.

We decided to insert the WSE Routing layer on the SOAP server layer. WSE routing is implemented through an IHttpHandler. Done this way, the handler can intercept the request for any of the client services and make routing decisions on behalf of the client.

So what’s the rub then? Don’t we have routing samples in the WSE QuickStart? Yes, we do, however, they introduce some complexity which makes the most critical requirement, not modifying the client proxies, nearly impossible. The routing samples assume your clients were built using WSE and introduce a requirement for WS-Address to be present in the client request (which can be modified through a SoapFilter). However, we can rebuild the same project using only classic Web Service SOAP standards which really cleans up the project and demonstrates how simple it is to utilize WSE 3.0 routing for your classic SOAP service needs! Download WSE_Soap_Router!

Components of the Sample:

• ClassicSoapClient – console application which was a web reference to DataCenter.asmx

• WSE_Routing_Sample – Classic ASMX web project with two web services:

• DataCenter.asmx – ClassSoapClient was built against this service

• DataCenter1.asmx –  this is where the call will be routed instead

• SoapRouter.cs – this is also contained in the WSE_Routing_Sample web project and contains the logic to process the incoming SOAP message and route where desired