General Service Testing Best Practices

Applies to: Windows Communication Foundation

Published: June 2011

Author: Alex Culp

This topic contains the following sections.

• Follow Consistent Naming Convention
• Assemble/Act/Assert
• Code Review Tests
• Create New Tests for Every Defect
• Use Test Categories
• Test as Part of the Build Process

Follow Consistent Naming Convention

Use a good naming scheme for your tests. A good scheme helps you to understand what is broken if a test fails. Before development starts, establish naming standards for the development team to follow. Also, it is much better to have a very long test method name then it is to have a test method that is not named well.

The following code tests the MyMath class, which is not shown in the example.

[UnitTest]
[TestMethod]
[ExpectedException(typeof(DivideByZeroException))]
public void Math_Divide_Throws_DivideByZeroException_When_Dividing_By_Zero()
{
    //assemble
    var math = new MyMath();

    //act
    math.Divide(1, 0);

    //assert - Assert is the exception
}

The method name may seem ridiculously long, but if this test fails, the problem is obvious.

The naming convention in this example follows the pattern: [Class Name]_[Operation Name]_[Expected Behavior]_[Scenario]. You do not have to follow this pattern exactly, but you should account for each aspect that this scheme highlights.

Assemble/Act/Assert

Remember that unit tests are code, too. It is as important to be consistent in how you develop the tests as in how you develop the application. A typical unit test has three steps. They are: assemble, act, and assert.

Assemble. This is where you create a new target object, and set up any mock objects that are needed to execute the test.

Act. This is where you actually invoke the target method.

Assert. This is where you validate the results.

It is a good idea to include a comment at each step in order to improve the readability of your test. The following code is an example of how to do this.

[UnitTest]
[TestMethod]
public void Math_Divide_Success_When_Dividing_By_NonZero_Number()
{
    //assemble
    var math = new MyMath();

    //act
    var result = math.Divide(2, 2);

    //assert
    Assert.AreEqual(1, result);
}

Code Review Tests

For automated tests to be useful, they must be treated as part of your code base. Code review the tests just as you do the application code.

Create New Tests for Every Defect

One of the most common problems in software development is a bug that you thought was fixed, and that reappears. After you identify the cause of the bug, create a test to verify the buggy behavior. Initially, this test should fail. Now, fix the bug and rerun the test. Ideally, the test should now pass. If you are use Microsoft® Visual Studio® Team Foundation Server, consider using the WorkItem attribute to associate the defect with the test that you use to verify it.

Do NOT use the WorkItem attribute on the code you are testing. This creates a dependency on a Visual Studio DLL that you probably do not want to deploy to production.

Use Test Categories

It is important to separate unit tests from integration tests. You can do this by project, folder/namespace, test categories, or a combination of these options. Test categories are discussed in more detail later in this article.

A simple approach is to use the TestCategory attribute and decorate your test methods. The following code is an example.

[TestCategory("UnitTest")]
[TestMethod]
public void SomeTest()
{
}

The drawback is that it is almost a certainty that a developer will type something similar to the following example.

[TestCategory("UnitTst")]
[TestMethod]
public void SomeTest()
{
}

As you can see, the category has a typo. Consequently, the test will not run as part of the Continuous Integration (CI ) build. (A continuous integration build is a build that runs any time a developer checks in code. It verifies that everything still compiles.)

A better solution is to create a pair of custom test categories. The following code shows how to do this.

public class UnitTestAttribute : TestCategoryBaseAttribute
{
    public override IList<string> TestCategories
    {
        get { return new List<string> { "UnitTest" }; }
    }
}
public class IntegrationTestAttribute : TestCategoryBaseAttribute
{
    public override IList<string> TestCategories
    {
        get { return new List<string> { "IntegrationTest" }; }
    }
}

You can now use one of the test category attributes, and not worry that a typing error will keep the test from being run. The following code is an example of how to include the unit test category.

[UnitTest]
[TestMethod]
public void SomeTest()
{
}

Some Useful Test Categories

Here are some useful test categories.

UnitTest. Use this category to denote unit tests. The Continuous Integration build can use this category to run only these tests. For more information, see later in this article.

IntegrationTest. Use this category to denote integration tests that will run as part of the nightly build process.

QAIntegrationTest. Use this category to denote integration tests that will be run when the services are deployed to the Quality Assurance (QA) environment.

LongRunningIntegrationTest. Use this category to denote integration tests that you might not want to include in typical builds because the tests take a long time to complete.

Note

Unit Tests and Integration tests are covered later in this document.

Test as Part of the Build Process

If you take the time to write unit and integration tests, you want them to be used. There is no way to know if your code, or even the tests themselves, have defects, unless you run them. To ensure that your tests are used, always engage management so that they know why the tests matter. Then, incorporate your tests into the build process. At a minimum, you must have a Continuous Integration build, and a nightly build.

The Continuous Integration Build

Set up the Continuous Integration build to compile the code and to run everything that has a [UnitTest] attribute. The build should fail if there are any broken unit tests. If the build times are short enough, and if you use Team Foundation Server, consider using gated check-ins. Gated check-ins prevent developers from checking in code unless the build succeeds. They help to ensure that people pay attention to the tests, and that other developers will not try to work with code that does not compile.

The Nightly Build

Nightly builds should build all of the code, and run all of the unit and integration tests. Turn code coverage on. Just as with the Continuous Integration build, the nightly build should fail if any test fails. A very helpful utility is the trx2html tool. It converts the TRX (test results) file into HTML. Include its results along with the code coverage reports, and build summary in the email you send to your team. For more information see trx2html.

Previous article: Introduction to WCF Testing Strategies

Continue on to the next article: Unit Testing: Introduction