Web Services: Concurrent Connections

The issue of concurrent outbound Web Service calls being throttled to two per endpoint comes up again and again with unmanaged and managed code – and it embarrassingly bit me last week in a BizTalk Performance and Scaling exercise despite me harking on about it all the time with standard .NET development L

The HTTP 1.1 Specification, specifies that a “single-user client SHOULD NOT maintain more than 2 connections with any server or proxy” . This is to ensure that a user cannot overload a remote Web Server, this is all understood and the various Microsoft HTTP stacks all respect this restriction as per the specification.

This is fine for user scenarios, but when you’re calling a Web Service from an ASP.NET Site, BizTalk Orchestration (via SOAP Adapter), etc. you will also hit this limit for the whole of your application rather than for each user of your application. This concurrent connection restriction is applied to an AppDomain which in the ASP.NET world means a Virtual Directory, which then means all users concurrently calling ASP.NET Pages will have any outbound Web Service calls throttled down to two concurrently, pending requests will then be queued.

This as you’d imagine causes a massive scalability and performance problem, and unfortunately this default behavior makes its way into live solutions more often that not, but fortunately it’s easy to fix.

In your code you can override this as per this article, in your initialization you need to set: ServicePointManager.DefaultConnectionLimit = 4;

This will then be hardcoded inside your application so my recommendation would be to control it via your .config file:

<system.net>

            <connectionManagement>

      <addaddress="*"maxconnection="40"/>

</connectionManagement>

</system.net>

More background on this here. As for the value you should use it’s really up to your scenario, as long as the remote Web Server can handle it (and is under your control) you probably want it as close to the number of possible threads available on the requesting machine. If it’s an ASP.NET page calling the Web Service you ideally want it set to 25*NumberofProcessors which matches the ASP.NET Worker Thread pool.

A quick and easy way to see if you’ve hit this issue is to monitor the number of Concurrent Connections to your Web Server, if you’re running IIS the “Concurrent Connections” counter under the Web Service performance object is the one to watch. If you never see this go about 2 * NumberOfRequestingMachines then you might be hitting the limitation. 

I was in the performance lab and we were only seeing 4 concurrent connections to the IIS Box hosting a Web Service, we had two BizTalk 2004 Servers (Quad Procs) and were putting a lot of load through the system and should have seen far higher connections. We put the above change into the BTSNTSVC.exe.config file and low and behold the number of Concurrent Connections got into the 30/40s J

For some reason I just assumed the BizTalk 2004 Install would “overcome” this limitation but it doesn’t, and the more I think about it – it shouldn’t either. The Installer doesn’t know how many connections your remote Web Service can handle – that’s something you need to tune.

Anyway, we live and learn – Hopefully this will help someone out there and save them some head scratching J 

If your calling Web Services from .NET, BizTalk Server 2004 and you have lots of concurrent users: Your almost certainly experiencing this problem!