Share via

The Tragedy of Thread Happiness Disease

  • Article

A JOS reader interested in developing server software asked recently

Is it possible to determine the number of concurrent threads a server can support from the server's specification?

Now, as I've said before, I'm no expert on performance tuning multi-threaded applications, but I have picked up a thing or two hanging around the real experts on the IIS team. In fact, the perf teams get questions in this vein all the time from customers, both internal and external.

The interesting thing about the question is that it is symptomatic of Thread Happiness, a peculiar disease which usually strikes programmers who are relatively new to large-scale multi-threaded software development.

How do I know that the questioner is getting Thread Happy? Because if they were writing an application where there were two threads or five threads or ten threads then they wouldn't be asking that question. They'd be asking "I want to write a server app with two/five/ten/whatever threads -- how buff does the server have to be?"

But they're asking the question "how many threads can I create on a given server?" so I can only assume that they are thinking of writing some application that is going to create a large and variable number of threads, as many as possible.

That's Thread Happiness right there. Don't do that! It's almost certainly a bad design. Massively multi-threaded applications cause more problems than they solve; they are very difficult to get correct, and even harder to get performant.

To actually answer the question, no, there's no way of telling that from the server spec. Why? Because there is nowhere near enough information in the "static" facts about the hardware. Server application threading performance depends on "dynamic" facts like:

  • What other processes are going to be running? 
  • What are those threads going to be doing? 
  • How much memory do you have free? 
  • How much contention, locking, busy waiting, context switching, blah blah blah, is there going to be?
  • What are the performance metrics that will determine when things have gotten too bad.

So, in short, don't even go there. Come up with a design that uses a small number of threads and tune that.

Now, you might be wondering "What about real-world server applications which do spin up a variable number of threads then? How do they decide how many is too many?"

It's quite hard. These things used to be configurable in IIS, but a few years back the IIS team had the realization that (a) you can't expect a human being to figure out what the ideal thread limit is, and (b) the ideal thread limit changes dynamically because of all the factors I mentioned above.

IIS therefore keeps a relatively small thread pool, and continually monitors its own performance, tweaking the thread pool count, thread priorities, etc, as conditions change. It tries to keep the server well-tuned dynamically.

This was non-trivial code to write. Though I've done quite a bit of optimization of software that runs in-process with IIS, the details of the IIS thread timing algorithms are way, way beyond my skills. I wouldn't recommend such an approach unless you've got a lot of experience in the field. A huge performance lab with a wide range of server hardware and a dedicated staff of experienced performance testers doesn't hurt either!

Comments

  • Anonymous
    February 15, 2004
    > It's quite hard. These things used to be configurable in IIS, but a few years back the IIS team had the realization that (a) you can't expect a human being to figure out what the ideal thread limit is, and (b) the ideal thread limit changes dynamically because of all the factors I mentioned above.

    It's unfortunate that the ASP.NET team hasn't figured this out as well:

    http://support.microsoft.com/default.aspx?scid=kb;en-us;821268

    Most users should never even care about implementation details like maxWorkerThreads, maxIoThreads, minFreeThreads, minLocalRequestFreeThreads etc.

  • Anonymous
    February 15, 2004
    Nothing new under the Sun:

    http://groups.google.com/groups?selm=01bbca48%24df50ef60%244a2fcbc7%40dns.brm.co.il&oe=UTF-8&output=gplain

    "If you are only targeting the new NT" ;-)

    OTOH consider that there are different environments that do espouse this sort of design, Erlang for example. They do have a very different implementation of threads, however.

  • Anonymous
    February 15, 2004
    The New NT? In 1999–2000 we first heard of Windows 2000 “based on New Technology Technology”, but it turns out that in 1996 you were already talking of New New Technology? :)

  • Anonymous
    February 16, 2004
    The complexity involved with synchronization isn't dependent on the number of threads; if it works for two threads, it sure will work with n. However, I agree that performance isn't directly proportional to the number of threads alone. Perhaps, a configurable pool is the best idea and if somebody is a bit above average (we call those coders "Rambo(es)" in our company), he should try writing a self-monitoring-and-adjusting server.

  • Anonymous
    February 19, 2004
    It's always struck me as a little bit odd that Active Server Pages, a web server, encourages developers to use VBScript and JScript to write server-side scripts. I mean, the whole point of a web server is that it produces complex strings as blindingly fast as possible, on demand.

  • Anonymous
    February 21, 2004
    Well, the situation has changed with .NET and it was never the "scripting" way in J2EE.

    I find scripting most beneficial when it comes to doing something quick and dirty with no compiler available.

  • Anonymous
    April 16, 2005
    The comment has been removed

  • Anonymous
    September 24, 2008
    It's always struck me as a little bit odd that Active Server Pages, a web server, encourages developers to use VBScript and JScript to write server-side scripts. I mean, the whole point of a web server is that it produces complex strings as blindingly

  • Anonymous
    September 24, 2008
    PingBack from http://blogs.msdn.com/ericlippert/archive/2003/12/01/speeding-can-slow-you-down.aspx

  • Anonymous
    October 28, 2008
    Well, I intended to spend the last three weeks blogging about C# design process in anticipation of our

  • Anonymous
    April 15, 2011
    Given 1) that dual, quad-core servers are common place now, 2) the Task Parallel Library portions of .NET 4, and 3) the time since the original post, does your advice of "come up with a design that uses a small number of threads and tune that" change at all?