The True Cost of .NET Exceptions

Here's an article under Jon's Homepage that was just brought to my attention.  It's an interesting analysis of (approximately) the raw cost of throwing an exception. Jon is definately right on that 200 exceptions per hour isn't gonna hurt anything.

However, there are some problems with the analysis at least in my opinion, because I believe it understates the true costs. I thought it would make a fun exercise to think about the approach taken because it is an interesting topic, and the techniques for thinking about the true cost are generally applicable.

So, asking Jon's kind forgiveness for picking on him, gentle readers, can you tell me why I would say this benchmark understates the cost?

Consider it a mini-quiz :)

Some other references that might be helpful:

• Exception Cost: When to throw and when not to
• When to create exception objects

My answers to follow.

Comments

• Anonymous
  September 14, 2006
  Well one thing is that it will cost developers a lot of time having to step through exceptions that are a part of regular control flow instead of only breaking in the debugger when a catastrophic failure occurs.
• Anonymous
  September 14, 2006
  A few things come to mind but they are probably outside of the scope of the "true cost" but this is a topic that interests me greatly so I am very curious about what specific things you are referring to.
  
  Here are my 2 cents...  
  
  1. The time taken doesn't include clean up costs of objects associated with throwing the exception
  
  2. The article discusses ApplicationExceptions. A common exception is the NullReferenceException, which has additional costs of throwing a native access violation exception
  
  3. If the exception is not handled directly in the function where they are thrown they may be rethrown at multiple levels. For example if you are throwing an exception in an ASP.NET app and it is not handled directly you would incurr costs of moving up the chain to the global exception handler.
  
  4. In the example we have one looping function. In the real world we would have multiple threads executing simultaneously. The added CPU usage may have a decremental effect on the process if we throw enough exceptions.
  
  5. Common coding practices for handling "unexpected exceptions" involves logging them to the eventlog or SQL server which makes the actual cost of the exception a lot larger.
  
  I have more real-life examples where the cost of an exception is much larger than the cost of just throwing it but I think I'll stop there:)
  
  
• Anonymous
  September 14, 2006
  The cost of an exceptions is dependant on the call stack before it, it's not fixed and higly dependant on the environment. Every rethrow of the exption multiplies it's cost proportionaly. So having a relatively low cost for a situation where and exception traverses a stack that only eventualy catches it might not be verry big. But an the cost of an excetion that traverses a stack where it's rethrown n times it has a cost of about n times in magnitude. Aso add a slight cost  for every specific catch blocks it traveses wihout catching it and also some costs for any finaly blocks that must be executed and the costs for creating container exceptions, but these costs are usualy neglijible compared to rethrowing.
  
  So I think the trap that you can fall in is the fact that your initial calculated cost can be multiplied without ever beeing aware of it!
  
  Another pitfall might be that the code that trows exceptions might find that in some circumstances the exceptional conditions are met way more often than originaly anticipated due to let's say an architecural change or data  shifts, in wich case something that wan't a problem before coud become a seriouse performace problem.
  
  Combining all this and I want to stay as far away from exceptions as posible!
  
• Anonymous
  September 14, 2006
  The glaring omission I notice is the cost to JIT the exception handling code throughout the callstack. If you're throwing exceptions from the same place all the time (as the tests do), you only pay once and may not notice that cost over the AppDomain's lifetime. But, if you're using exceptions inappropriately throughout your code, you're paying all the associated costs with JITting the exception handlers, keeping the EH pages in memory (and the opportunity cost of paging out data or other "warm" code).
• Anonymous
  September 14, 2006
  The comment has been removed
• Anonymous
  September 14, 2006
  The comment has been removed
• Anonymous
  September 14, 2006
  Things that have caught my attention:More Information about Zune has flooded the web (neowin has some
• Anonymous
  September 15, 2006
  If you know something causes say 200 exceptions a hour then well that's hardly Exceptional? More like a lack of error handling.
• Anonymous
  September 15, 2006
  I would hazzard a guess and say that perf really suffers when you decide what you're going to do when you handle the exception. Eg. If you automatically send the error report to some central location (via webservice, email etc..), or just dump it in a lof file,  those costs are WAY higher than anything to do with the actual throwing of the exception.
  (Good design would take care of this, use another thread and do all your reporting async.)
• Anonymous
  September 15, 2006
  The comment has been removed
• Anonymous
  September 15, 2006
  Its Pretty obvious if you have good coding and design this should definetly not be the problem, and even that you should do you exceptions asyncrounsly.
• Anonymous
  September 15, 2006
  The comment has been removed
• Anonymous
  September 15, 2006
  Ian is wise :)
• Anonymous
  September 15, 2006

1. Move the "new ApplicationException()" outside the loop. This will take the GC out of the picture.
   
   2) Raise the thread prioriy to Highest. This lowers the "overhead" of other threads.
   
   These 2 improvements got me a factor of 3 improvement.

• Anonymous
  September 16, 2006
  I think this scenario is rather misleading. On my system (an AMD Athlon 64 2.2 gHz Server 2003, SP1) on .NET 2.0 (32 bit, release build) this benchmark produces about 29,000 exceptions/sec.  This means it takes on average about 35 microseconds to throw/catch the exception.
  
  Compared to an hour, that's tiny. But compared to returning an error code, it's about 35,000 times more expensive.
  
  Compared to simple system calls (eg. SetEvent, it's about 70-100 times more expensive.)
  
  It's also several times slower than writing 128 bytes to a file (where the file write is completed in the FS cache).
  
  
  To evaluate the 200/hr scenario, one has to ask several other questions.  Do these 200/hr happen to coincide with the 200 transactions/ hr that the application processes?  If so, what is the time with and without the exception. If it takes 35 us to process a tranaction without the exception, and 70 with the exception, it's doubled your processing time.   That's a big difference.
  
  I don't that the 200/hr sheds much light on the cost of exceptions.

• Anonymous
  September 17, 2006
  Hi Rico,
  
  I agree that this test is too simple since the exception creation is very cheap compared to the stack unwinding. A much more realistic test would throw an exception which is catched two or three functions above to let more stack unwinding happen.
  
  Yours,
    Alois Kraus
  

• Anonymous
  September 18, 2006
  The comment has been removed

• Anonymous
  September 18, 2006
  David's magnum_opus of a response is full of excellent points.  While I can't say I agree with all of them the spirit seems right on to me and you'll find my points of disagreement to be not very important compared to the points of agreement when I post my own thinking on the matter (probably later today).
  
  
  

• Anonymous
  September 18, 2006
  I agreed that the test is not a good measurement in the real world. Some of the problems I see:
  
  1) the call stack is only one. Therefore the cost of rewinding is not included.
  2) some of the other costs may not be reflected. For example, the fianlizer may be called during the rewinding.
  
  

• Anonymous
  September 21, 2006
  The comment has been removed

• Anonymous
  September 25, 2006
  Well, once again my elite readers have made a solution posting by me nearly redundant.  There were...

• Anonymous
  October 09, 2006
  The comment has been removed

• Anonymous
  December 21, 2006
  I had an interesting email conversation here at work recently and as usual I can't share it all but I

• Anonymous
  December 21, 2006
  Well, once again my elite readers have made a solution posting by me nearly redundant. There were many

• Anonymous
  January 01, 2008
  PingBack from http://imehta.com/blog-ketan/?p=15