Test-Driven C#

Improve the Design and Flexibility of Your Project with Extreme Programming Techniques

Will Stott and James Newkirk

Code download available at:ExtremeProgramming.exe(127 KB)

Contents

Why Designs Degrade
A Simple Introduction
Lessons Learned
Automating Your Testing with NUnit
Taking NUnit for a Test Spin
Test Cases
Test Fixtures and Suites
Using TDD in the Real World
Developing a Test and Creating a Library
Creating the GUI
Conclusion

Wouldn't you just love to create code that becomes easier to work with as the project matures rather than more difficult? It seems that no matter how much care you take, sooner or later, your code will become a maze. The bigger the project, the worse this problem gets. How many times have you started a project with a near-perfect design only to see it unravel once coding starts?

Test-driven development (TDD) alters the process of writing code so that change is not only possible, but desirable. Development revolves around three basic activities: writing a test, writing code to pass the test, and refactoring the code to banish duplication to make it simpler, more flexible, and easier to understand.

This cycle is repeated frequently, each time running all the tests in order to ensure that the product is kept in a working state. The long gaps between the design, coding, and testing phases are gone, thus making for a much better learning environment. Therefore, your design (and code) actually improves rather than degrades as the project matures.

What makes TDD so effective is the automation of your programmer (unit) tests, and the great news is that the only tools you need are available for free on the Internet. These are not reduced functionality versions of a commercial product, but high-quality software made available by fellow developers. This article explains how you can obtain and use NUnit to practice TDD for development with C# (or any Microsoft® .NET Framework-based language). Note that similar tools are available for C++ and Java developers, as they are for most other languages and operating systems. The ready availability of such tools gives TDD a universal appeal which, combined with its close association with extreme programming, has done much to encourage its use.

Why Designs Degrade

Most traditional software processes are based on the assumption that you can get the design right in the beginning and then pass it through the development machine to generate a perfect product. This is a production-line mentality that values uniformity and minimizes variation. Such processes don't value communication and feedback in the same way that TDD does, so they are less effective at both generating information (failing tests) and allowing people to learn from it (fixing the design).

Why can't you get the design right from the beginning? Because at the start of the project you have imperfect knowledge about the software you're developing. Iterative development acknowledges this fact and helps you identify significant problems earlier in the project rather than leaving them all to be discovered at the end. However, the iteration will not be canceled to let you return to the design phase to address a simple problem like a badly named public class. Nobody wants to hear such minor concerns and the process is unfortunately often designed to suppress them because the cost of moving between phases is too high.

The accumulation of these small issues causes the real difficulties in many traditional development processes. You will always have more important priorities than making changes that offer no functional advantage. However, the longer the badly named class remains in the code base the more prevalent its usage becomes, and the bigger the task of changing it. After a while, the team will begin fixing such problems unofficially while coding, so soon you will be planning an entire release that does little else but attempt to make the code correspond to the design documentation, or vice versa. At this stage, your up-front design is worthless—it is the code that speaks.

Test-driven development allows you to defer decisions until you understand the problem better. You don't need to produce a perfect architectural design up front at a time when you have the least knowledge about how the product will develop. This challenges most established ideas about software development and is to some degree counterintuitive, so we suggest you try TDD with an open mind and discover its power for yourself.

There are also benefits to TDD that are much easier to grasp. It encourages communication by making your programs self-documenting—the set of tests you develop show how the code works in a way that a manual can't. It encourages feedback by asking you to view your code from the perspective of someone writing a test for it, which helps you create objects and components that are more loosely coupled and cohesive. It encourages simplicity by letting you defer big decisions so you can concentrate on getting the little ones right. Finally, it gives you the courage to push the boundaries of your code by giving you a set of tests that tell you immediately when something breaks.

A Simple Introduction

The only way to really understand TDD is to do it, so let's start with a very simple example that doesn't require any special tools. We're going write a small program to help us plan a rectangular lawn for a home, but before we start there are two things we will want to test: calculate the area as 6 units when the length is 3 units and the width is 2 units, and also calculate the perimeter as 10 units for a length of 3 units and width of 2 units.

Writing down these tests helps focus our attention on what is important in our program. In this case, it seems reasonable to develop an object that will help model the problem domain, so we'll create a class called Quad, named for the quadrangular lawn we're trying to construct, and instantiate it as an object in a simple console application. Here are the steps to follow along:

1. Start up Visual Studio® on your desktop, select File | New | Project, then choose "Console Application" from the list of C# project types. Type "QuadApp" into the Project Name box and press OK.
2. Type into the member function Main a few lines of code to create an instance of Quad and then assert that when the object Area method is given a length of 3 and width of 2, it returns the value 6:

static void Main(string[] args) { Quad q = new Quad(); System.Diagnostics.Debug.Assert(q.Area(3,2) == 6); }

1. Select Project | Add Class, type the name "Quad.cs" into the dialog box, and press Open to create the class.
2. Type the code required to give Quad a member function Area as required by its use in Main:

public class Quad { public int Area(int length, int width) { return 0; } }

1. Select Build | Build QuadApp from the menu and this time the build should succeed. However, when you run the program (Debug | Start) it asserts because Area doesn't return 6.
2. Hardcode a return value of 6 for the function Area, rebuild the program, and run it again. This time the program runs without asserting.
3. Improve the implementation and design of Area by returning the product of its input parameters rather than hardcoding a return value. Check that you haven't broken anything by rebuilding the program and rerunning the test.

The second test (calculate the perimeter as 10 units) works in a similar way. Start by writing the test using assert in Main and then follow Steps 4 to 7, but for Perimeter rather than Area. When you're trying to improve the design (Step 7), you will probably reach the conclusion that perhaps you should be passing the length and width to the constructor and storing them as properties of the object rather passing them as parameters to the member functions. Change Quad so that it stores length and width as properties (shown in Figure 1), rebuild the program, and then run it to make sure that you haven't broken anything.

Figure 1 Quad After Some Refactoring

public class Quad { private int m_length; private int m_width; public Quad( int length, int width) { m_length = length; m_width = width; } public int Area() { return m_length * m_width; } }

We have just completed an exercise that demonstrates TDD in its very simplest form. Here's a summary of each step involved:

Write a test that fails We selected the test that seemed the easiest to implement (although in this case they were all simple) and wrote it. Next, we wrote the simplest implementation of Quad in order to get the program to compile. However, when we executed the program, it asserted because Area didn't return the value 6.

Fix the code so you pass the test To fix the code, we performed the simplest task and hardcoded a return value of 6. The objective was just to get the program working so it passed the test. However, the next test might use different parameter values, forcing us to implement a proper algorithm in order to pass both tests, even if we didn't refactor as described next.

Refactor the code Since we achieved the correct observable behavior in our program, we tried to make the code more maintainable by removing duplication, making it simpler, more flexible, and easier to understand. We checked that this change hadn't altered the desired behavior of the program by rerunning the test.

The idea of reworking your code to improve its maintainability without changing its observable behavior is not new. However, there is more to refactoring than just tidying up your code from time to time. Refactoring must become a key part of your software development activity, done in a methodical way using appropriate tools so as to progressively improve the quality of your code. The aim is to achieve perfectly maintainable software by performing many small refactorings of the code throughout the product's life.

Refactoring is an essential step in test-driven development, providing the necessary feedback that allows your design (and code) to improve as the product develops. You can, however, refactor without performing TDD and still banish duplication while making the code simpler, easier to understand, and more flexible. You just need to make refactoring a regular activity in whatever development process you are following and ensure that your changes are validated with appropriate regression testing.

The power of refactoring lies in its ability to reduce the danger inherent in making changes to working code. Good tools can help address some of the potential perils, but equally important is taking a series of small steps and having a structured and disciplined approach. Martin Fowler's book Refactoring: Improving the Design of Existing Code (Addison-Wesley, 1999) gives a good introduction to these issues by providing a useful catalogue of patterns for refactoring and guides you through the refactoring process by offering several relevant examples.

At present, there is not much support for refactoring in Visual Studio .NET, apart from using Find and Replace. However, more powerful tools are forthcoming so that you can do smart things like symbolic renaming, and changing the name of a class at the level of the compiler-generated parse tree rather than just performing textual substitution in your source files. You can look forward to the day when you can select a piece of code, apply a refactoring pattern from a menu provided by Visual Studio, and then continue with your work—confident that you've just made the program more maintainable without introducing any bugs or changing its observable behavior.

Lessons Learned

The most important lesson you should have learned so far is that TDD is simple. In fact, TDD is a great for obtaining a better understanding of any new language, technology, or component that comes your way. It allows you to decide the size of steps you need to take. An expert might have taken bigger steps and, using his experience, avoided some of the intermediate steps that we took. When things start to go wrong, however, you always have the option of going back and doing it again, taking smaller steps. There are number of other things you may have noticed about TDD:

• The tests document the code. We can see exactly what Quad does by looking at the tests.
• We can measure progress by the tests passed. Each piece of functionality is proved by a test, so anyone can run the tests to see that claims of progress are well founded.
• The tests make everyone confident about changing the code. Even though you might be a novice C# programmer working on a class written by an expert, running all of the tests will reassure you that you haven't broken anything.
• Mistakes, like passing the length and width as parameters to Area and Perimeter rather than to the constructor, are corrected by refactoring.
• When you apply TDD in the real world, you're going to generate lots of tests, so you need to organize your testing using a tool like NUnit.

Automating Your Testing with NUnit

A TDD project might generate thousands, or even hundreds of thousands, of tests. Given that everyone on the team will be going through the "write a test, fix the code, refactor" cycle frequently, it's essential that you can both write and run tests efficiently. A test framework like NUnit is designed to help you do this. It allows you to arrange your test cases into individual projects that can be loaded into NUnit in the same way that you create projects in Visual Studio. You can also display all the project's test cases in a hierarchy, run tests individually or as a suite, and see the result of their execution as pass (green) or fail (red) with detailed information about each failure (see Figure 2).

Figure 2** Test Status in NUnit **

Additionally, NUnit provides special functions you can execute at the start of a test suite and upon its completion to initialize and clean up "test fixtures" (such as log file, database connection, comm link) shared by various test cases. You can also define a Setup function to be run before a test case is executed and a TearDown function to be run after it completes. This helps you isolate test cases from each other by resetting the system between each one.

NUnit lets you write a test case in the same language and environment as your application. With no special test language to learn, developing tests is quick and straightforward. There is also a fully featured Assert class that will provide information about why a particular test failed.

Finally, NUnit can be run as a console application that simply writes its results to the command prompt. This allows you to automate a formal build process so you can, for example, rebuild the application overnight and run the full suite of tests. It is also possible for the NUnit console application to produce an XML log of the test results.

People with a background in software testing will appreciate the value of these features, but even if you don't have this sort of experience, it should be clear that NUnit has much to offer you as a developer (see the sidebar "Testing Terminology"). Nevertheless, expert or novice, the best way to find out how NUnit helps you organize your testing is to try it out. You can download NUnit from https://www.nunit.org. Precise details of what you are permitted to do with the product are given in its license file.

I used the NUnit version 2.1 beta release for this article, but you should download the MSI file for the latest production release (approximately 1.5MB). Once this file is on your PC, you just need to double-click it to start the Windows® Installer. It works with both version 1.0 and version 1.1 of the Microsoft .NET Framework, though you should refer to the NUnit documentation for more information on system requirements.

After installing NUnit, select the submenu called Test in order to run the suite of tests that confirm that the product is working properly. Just click the menu option, wait for NUnit to open on your desktop, and then press the Run button.

After less than a minute, all the nodes in the treeview should be green, thus indicating that the tests have been executed and passed. The number of tests run is shown in the status bar, together with the time taken to execute the tests, and the number of failures. In the unlikely event that you get some red nodes, indicating failed tests, you may need to reinstall the product or get some advice from the NUnit.org site.

Taking NUnit for a Test Spin

NUnit comes with a number of samples to get you started, so let's walk through a development episode using one of them, namely, the Money project.

Start up Visual Studio on your desktop, select File | Open | Project, and select the file Money.csproj, which is in the NUnit installation directory (Program Files\NUnit) in the subdirectory src\samples\money. Press Open and wait for the project to load in Visual Studio. In the Solution Explorer window, look in the References folder and check for nunit.framework. Build the project Money by choosing Build | Build Money. The build should succeed without any errors or warnings.Testing Terminology

Test case (Test) A unit of test in the same way that a function is a unit of code. A test case usually contains the test object, the various fixtures it needs, and the statements and assertions that make up the test.

Test environment The (known) state of the system upon which you are running the tests. It is usually a simpler and more controllable version of the production environment in which the product will eventually operate.

Test fixture (test scaffolding) An object shared by one or more test cases concerned with initializing the test object or providing it with some resource.

Test framework A library that facilitates the writing and running of test cases, such as the various classes provided by nunit.framework.dll.

Test object The object being tested, typically an instance of the class you're developing.

Test script A file containing one or more test cases and any other information that may be required for creating a test program, such as the source code files of the test program.

Test suite A collection of test cases (or suites) that are associated with a particular component, assembly, or area of the product. When you run a full test suite, you are running all the application's test suites (the test cases).

Test tool An application concerned with running test cases and suites, such as the NUnit application.

Start up NUnit on your desktop. The treeview should be empty—if not, close any existing project by selecting File | Close. Open the file Money.csproj by choosing File | Open in much the same way you opened it in Visual Studio. The treeview should now contain a collection of test cases.

Select the root node in the treeview (the file Money.csproj) and click the Run button in NUnit's right-hand pane. The full suite of test cases will then be executed and you should have just one failing test—MoneyBag Equals.

Select the test case MoneyBag Equals (the red leaf node) and locate the problem line in each file. This information is given in the lower part of NUnit's right hand pane, shown in Figure 3.

Figure 3** NUnit Error Tracing **

Switch to Visual Studio and open the offending file at the appropriate line. Comment out the line (insert // at the start of the line) and rebuild the program (Build | Rebuild Money). The build should succeed without any errors or warnings.

Now switch to NUnit and note that the treeview's nodes are all grey, indicating that the test cases have not yet been run for the program you've just updated. Then select the root node in the treeview and click the Run button again. You should observe that all the tests are running without failure.

This is just about all there is to working with NUnit. You are now ready to start writing test cases for your own projects.

Test Cases

The test case applies a defined set of inputs and also measures a de-fined set of outputs in order to validate that some program characteristic has been implemented without introducing any bugs. This is true for any kind of test that you might want to write.

First of all, a test case should concern itself with validating just one aspect of the program's behavior and it should either pass or fail. A test may fail because of a bug in the product, but it can also fail due to a bug in the test itself—a false negative. Similarly, a test may pass due to a bug in the test rather than because the product has behaved correctly—a false positive. Good tests should obviously avoid false negatives and should never yield false positives. In addition, a good test suite will not only test valid scenarios but will also test error cases.

The test object and its environment should always be in the same state before a test case is executed so that it can be repeated as often as necessary and give consistent results. When a test case fails you should be given adequate information about the location of the failure and its reason so that the problem can be corrected, which is why you want each test case to validate just one thing.

Now let's apply this theory by writing a few simple test cases for the Quad example. Open the Quad project in both Visual Studio and NUnit the way you did for the Money project—select File | Open and locate QuadApp.csproj. Add to the project the reference to nunit.framework by selecting Project | Add Reference, browsing for the file nunit.framework.dll (located in the NUnit installation directory within the subdirectory bin), and pressing OK.

Use Visual Studio to create the class QuadTest: select Project | Add Class, enter the name QuadTest.cs, and press Open. Enter the code shown in Figure 4 into QuadTest, build it (Build | Build QuadApp), and then run the tests using NUnit (press Run). Click the Console.Out tab in the right-hand pane to see the execution order.

Figure 4 QuadTest Class

using System; using NUnit.Framework; namespace QuadApp { [TestFixture] public class QuadTest { [TestFixtureSetUp] public void DoTestSuiteSetUp() { Console.WriteLine("TestSuiteSetUp"); } [TestFixtureTearDown] public void DoTestSuiteTearDown() { Console.WriteLine("TestSuiteTearDown"); } [TearDown] public void DoTestCaseTearDown() { Console.WriteLine("TestCaseTearDown"); } [SetUp] public void DoTestCaseSetup() { Console.WriteLine("TestCaseSetup"); } [Test] public void Area() { Quad q = new Quad(); Assert.IsTrue( q.Area(2, 3) == 6); Console.WriteLine("Area"); } [Test] public void Perimeter() { Quad q = new Quad(); Assert.IsTrue( q.Perimeter(2, 3) == 10); Console.WriteLine("Perimeter"); } } }

You can now see how the various Setup and TearDown functions might allow you to control the test environment so that your test cases can be repeated as often as necessary and give consistent results. These functions help you ensure that the running of one test case doesn't affect another (test case isolation). A good proof of test isolation is the ability to run your test cases in any order. Try changing the order of your test cases to confirm their isolation.

Test Fixtures and Suites

An NUnit test fixture is an object shared by one or more test cases concerned with initializing the test object or providing it with some resource. In terms of NUnit, a test fixture is a class with the attribute [TestFixture] whose methods provide:

• [Test] functions that form the various test cases. These test cases should be atomic operations and should have no dependencies on other tests.
• [SetUp] and [TearDown] functions needed to reset the environment between test cases. They are each run once at the beginning and end of each test.
• [TestFixtureSetUp] and [TestFixtureTearDown] functions required for objects shared by test cases. They are each run once at the beginning and end of the test fixture.

Any objects that you need to run the tests can be created as instance variables of the [TestFixture] class (shared by all test cases) or local variables of the methods (private to an individual test case), and this is why it is described as a test fixture. However, you can also think of NUnit's [TestFixture] as a way of organizing a test suite as each class forms an individual branch in the treeview under the name of the project.

You should now be able to start practicing TDD as you can write suites of tests for your own projects and can use NUnit to run them. Let's look now at some of the issues you might encounter doing more serious development work.

Using TDD in the Real World

One of the first things to consider when using TDD since a commercial project is deciding how to organize your programs so that the production code can be easily separated from the code used to test it. You want to run your programmer tests throughout the product's development and yet be able to easily remove them for the purposes of releasing the code.

Another issue you might encounter is the difficulty of testing GUI applications driven by mouse and keyboard input. For example, how do you write a test that simulates a user clicking a dropdown list and then verifies that it is populated with a given list of country names?

The answer to both of these problems lies in dividing the code into appropriate components that can be built, tested, and deployed separately. For example, rather than building Quad as a class contained in the same executable file as the main application, it could have been contained in a separate library (.dll). This would have allowed us to develop both the test program and the domain program as separate executables (.exe) that shared a common library (.dll) containing Quad. Note that if you're only building a library, NUnit and your test suite can be used as the interface, rather than requiring a separate harness.

The idea of keeping the main program very simple and putting the business complexity into classes contained in a library can also help solve the problem of testing GUI applications. One of the rules of TDD is that you don't test third-party code, so there is no requirement for you to test the GUI framework classes, although it's sometimes useful to test interfaces, particularly if they're a bit flaky. This means you can catch the user event in a class that you know works and then pass it through for processing to the class you're developing. Again, you can separate the test program and the domain program into separate executables sharing the common libraries that form the bulk of your development effort.

Let's see how this might work in practice by using TDD to develop a combobox list for a simple forms application.

Developing a Test and Creating a Library

Create a new console application called CountryTest by selecting File | New | Project in Visual Studio. Add to the project the reference to nunit.framework as you did when writing the first test case. Create the class CountryBoxTest by selecting Project | Add Class, entering the name CountryBoxTest.cs, and pressing Open. Enter the code shown in Figure 5 into CountryBoxTest and attempt to build it. The build fails because the class CountryList hasn't been written yet.

Figure 5 CountryBoxTest Class

[TestFixture] public class CountryBoxTest { [Test] public void CheckContent() { CountryLib.CountryList list = new CountryLib.CountryList(); Assert.AreEqual("UK", list.GetCountry(1)); Assert.AreEqual("US", list.GetCountry(2)); Assert.AreEqual("CH", list.GetCountry(3)); Assert.AreEqual(null, list.GetCountry(4)); } }

Use Visual Studio to create a new class library called CountryLib by selecting File | New | Project (add to the existing solution). Create the class CountryList by selecting Project | Add Class, entering the name CountryList.cs, and pressing Open. Enter the following code into CountryList and build the library:

public class CountryList { public String GetCountry(int No) { return null; } }

First add to the project the reference to CountryLib.dll and rebuild the console application, CountryTest. This time it works because the CountryList has now been written.

Open the project CountryTest.csproj in NUnit ( File | Open) and run the test. It fails because CountryList.GetCountry returns null. Switch to Visual Studio and in CountryList fix GetCountry so it returns the values required by the test—just add some initialized strings as instance variables, as shown in Figure 6. Do the simplest possible thing to get the program working. Rebuild the CountryList library, switch to NUnit, and then rerun the test—it should now run successfully.

Figure 6 CountryList After Some Refactoring

public class CountryList { string one = "UK"; string two = "US"; string three = "CH"; public string GetCountry(int No) { if ( No == 1 ) return one; if ( No == 2 ) return two; if ( No == 3 ) return three; else return null; } }

Do any refactoring of CountryList that you think is necessary and rerun the test to check that you haven't broken anything. You might think that storing the strings in an ArrayList would make the code simpler, easier to understand, and avoid duplication, but you might also choose to defer this decision for now.

Creating the GUI

Use Visual Studio to create a new Windows-based application called CountryApp by selecting File | New | Project (add to the existing solution). Add to the project the reference to CountryList.dll. Next, add a combobox to the application by dragging one from the toolbox onto the main form. Enter the code shown in Figure 7 into CountryApp and build the application.

Figure 7 CountryApp Class

private void InitializeComponent() { ... CountryLib.CountryList list = new CountryLib.CountryList(); String country; int index = 1; while ((country = list.GetCountry(index++)) != null) this.comboBox1.Items.Add(country); ... }

You now have the basic organization for your project that will allow you to create the GUI application by developing your CountryList library using TDD techniques. Think about how CountryList could process user interface events (mouse click, keyboard) in a way that could be both tested by CountryTest and used by CountryApp. You will probably come up with a much better solution for CountryList than the one presented here, which is precisely the point we want to make: test-driven development enables the design to improve as it matures. You don't need to produce a perfect design up front, which is good news for developers.

Conclusion

Most designs are created from the top down and are essentially an exercise in understanding and solving a problem by classifying things into categories based on observable traits—in other words, we try to create a hierarchy of objects that model the problem domain. In contrast, TDD is performed from the bottom up by sequentially applying a series of simple solutions to small problems that eventually evolves into a design. Refactoring ensures that the design converges toward a good solution rather than diverging away from it. If you think this is radical, you're right. TDD has the potential to change the way software is developed in this decade in much the same way object-oriented techniques did in the last.

A note on NUnit: when using Visual Studio to open some of the samples and pre-built tests that are distributed with NUnit, you may discover that the reference to nunit.framework.dll is shown as "not found," so look in the Solution Explorer window, References folder. In this case you need to delete the existing reference (select the file, right-click | Remove) and then add it again (select References, right-click | Add Reference, browse to locate the file, and press OK). The file nunit.framework.dll can be found in the NUnit installation directory within the subdirectory bin.

Will Stott is a freelance consultant living in Zurich, Switzerland. He is also an associate of Exoftware, a Dublin-based software company specializing in the introduction of extreme programming into organizations. Reach him at wstott@exoftware.com.

James Newkirk is the development lead for the Microsoft Platform Architecture Group team. He is the co-author of Test-Driven Development in Microsoft .NET and Extreme Programming in Practice. He can be reached at jamesnew@microsoft.com.