Share via

Measuring managed code quickly and easiliy: CodeTimers

  • Article

My performance blog entries to date have been 'foundational'.  In entries so far, I talk about how to use Visual Studio to look at the native code generated for the runtime.   With this foundation, we can now start exploring what the native code for managed code looks like and what optimzations the runtime does on your behalf.

However before we do that, we need one more foundational piece.  You really can't do performance work, without measuring, and so we need tools to help us measure.   The simplest of these is the stopwatch.   Thankfully Version 2.0 of the rutnime provides a nice, high resolution class that does this called System.Diagnostics.Stopwatch, which is ideal for measuring short intervals accurately. 

As nice as System.Diagnositcs.Stopwatch is, I found myself wanting more.  I was repeatingly typing code to start and stop the Stopwatch, subtracting the difference, subracting out the harness overhead, and doing statistics on the resulting numbers.    This led me to write two classes to automate this, call CodeTimer, and MultiSampleCodeTimer.   I have included the source for these in the Zip file below.  Simply open it, copy the directory to your local drive, and open the .SLN file using visual studio.   You can even compile the code without having Visual Studio.  Simply run the buildIt.bat batch file  The only prerequisite is that you need V2.0 of the .NET runtime installed (available from windows update, among other places). 

The basic usage pattern for CodeTimer is 

     CodeTimer timer = new CodeTimer();
    timer.Measure("Measurement Name", delegate {
       // Code that you want to measure
    });

Basically I am making use the 'anonymous delegate' feature of c# to allow the client to specify the code that he wants measured inline.  One of the nice feature of anonymous delegates is that they 'capture' any local variables that are in scope.  Thus you can write things like

    
    CodeTimer timer = new CodeTimer();
    string myString = "aString";
    string outString;
    timer.Measure("Measurement Name", delegate {
       outString = myString + myString;  // measuring concatination.  
    });

To measure the time it takes to concatinate two strings.  What is actually going on 'under the hood' is that the C# compiler generates a class with two fields 'myString' and 'outString' (a field for every local variable referenced in the body of the delegate code). An instance of this class is created and initialized to the values of locals and this object is used to create a delegate to pass to 'Measure'.  Thus the locals are 'captured' and passed to the code that 'Measure' will ultimately call.   

This detail is imortant if you are measuring very small operations, because field access generates different code than local variable access.   Thus in the example above, I am not just measuring string concatination, but also fetching an instance field (mystring) twice, and setting an instance field (outstring).   Generally these fetches are small, and so they don't purturb the data much, but you should be aware of them. 

When you are measuring very short times (like the case above), you are likely to wan to run the test in a loop.  CodeTimer has built-in support for this.   I can run the example above 1000 times in a loop by doing

    
    CodeTimer timer = new CodeTimer(1000);
    string myString = "aString";
    string outString;
    timer.Measure("Measurement Name", delegate {
       outString = myString + myString;  // measuring concatination.  
    });

Which tells 'timer' to run any measurment 1000 times.  This is great for measuring micro-operations.  

While 'CodeTimer' is great, I want still more.  I don't want just one sample, but 10 or more, so I can take the average and standard deviation and understand how 'noisy' my data is.  That is exactly what MulitSampleCodeTimer does.   What is nice is that only the defintion of timer changes. 

    
    MultiSampleCodeTimer timer = new MultiSampleCodeTimer(10, 1000);
    string myString = "aString";
    string outString;
    timer.Measure("Measurement Name", delegate {
       outString = myString + myString;  // measuring concatination.  
    });

Which asks that the code be run 1000 times per measurment and that 10 measurments be taken and used to compute min, max, mean, median and standard deviation.  The results are then printed (we are reallly starting to add value). 

I find I use MultiSampleCodeTimer alot.   I use it in the'TypePerfMeasurement' project below to measure the speed of some reflection type constructs.   More on that in my next blog entry. 

SUMMARY:

In this blog entry, I have introduced to two classes 'CodeTimer' and 'MultiSampleCodeTimer' that can be used to meause the performance (time), of a wide variety of code.  You are encouraged to open, copy and build the attached sample below (you don't need VS to do it just the .NET runtime), and try it for yourself.   Measuring performance has never been easier.

Next time we will talk about the numbers that the TypePerfMeasurement experiment actually produce and talk about why the performance is the way it is. 

HOW TO USE THE DEMO:

I want my blogs to be 'hands-on' which means that there should always be something you can do after reading to experiement with the concept I talked about (that is there is homework!).  To make this easy I have attached a ZIP file that is a archive of a Visual Studio Solution for code to experiement with.  Note that you do NOT Need Visual Studio to compile and run the demo, so don't let that stop you.  To unpack this simply

  1. Click on the ZIP file attachement at the bottom of the post.  This will bring up a dialog asking if you wish to open the file. 
  2. Click on the 'Open' button.   This will bring up an explore window displaying the contents of the ZIP file, which is a single directory.  If you just want to look at files, you can browse the files directly from this window. 
  3. Open a target folder on your local computer.  This can be anywhere, but if you have Visual Studio installed 'My Documents\Visual Studio 2005\Projects' is a good place
  4. Drag the directory from the ZIP file window into the target folder.  This copies the directory and all its contents to the local hard drive. 
  5. Once copied, you can close the ZIP window, and click into the target directory.  There you will find a .SLN file, which you can double click on to bring up the solution in visual studio. 
  6. If you don't have Visual Studio, you can still build the solution, there is a one line batch file called 'buildIt.bat' file that does this (it simply turns around and calls MSBuild to do its work).  You do need Version 2.0 of the .NET Framework installed.  If you don't have it you can get it from www.windowsupdate.com (custom downloads), or here. 
  7. All the samples build and run and do something interesting 'out of the box',  They are meant to be easy to understand, commented, samples so start looking around and experimenting (Hitting F10 to start stepping into the code is a good place to start). 

TypePerfMeasurement.zip

Comments