Use code coverage for unit testing

Important

This article explains the creation of the example project. If you already have a project, you can skip ahead to the Code coverage tooling section.

Unit tests help to ensure functionality and provide a means of verification for refactoring efforts. Code coverage is a measurement of the amount of code that is run by unit tests - either lines, branches, or methods. As an example, if you have a simple application with only two conditional branches of code (branch a, and branch b), a unit test that verifies conditional branch a will report branch code coverage of 50%.

This article discusses the usage of code coverage for unit testing with Coverlet and report generation using ReportGenerator. While this article focuses on C# and xUnit as the test framework, both MSTest and NUnit would also work. Coverlet is an open source project on GitHub that provides a cross-platform code coverage framework for C#. Coverlet is part of the .NET Foundation. Coverlet collects Cobertura coverage test run data, which is used for report generation.

Additionally, this article details how to use the code coverage information collected from a Coverlet test run to generate a report. The report generation is possible using another open source project on GitHub - ReportGenerator. ReportGenerator converts coverage reports generated by Cobertura among many others, into human-readable reports in various formats.

This article is based on the sample source code project, available on samples browser.

System under test

The "system under test" refers to the code that you're writing unit tests against, this could be an object, service, or anything else that exposes testable functionality. For this article, you'll create a class library that will be the system under test, and two corresponding unit test projects.

Create a class library

From a command prompt in a new directory named UnitTestingCodeCoverage, create a new .NET standard class library using the dotnet new classlib command:

dotnet new classlib -n Numbers

The snippet below defines a simple PrimeService class that provides functionality to check if a number is prime. Copy the snippet below and replace the contents of the Class1.cs file that was automatically created in the Numbers directory. Rename the Class1.cs file to PrimeService.cs.

namespace System.Numbers
{
    public class PrimeService
    {
        public bool IsPrime(int candidate)
        {
            if (candidate < 2)
            {
                return false;
            }

            for (int divisor = 2; divisor <= Math.Sqrt(candidate); ++divisor)
            {
                if (candidate % divisor == 0)
                {
                    return false;
                }
            }
            return true;
        }
    }
}

Tip

It is worth mentioning that the Numbers class library was intentionally added to the System namespace. This allows for System.Math to be accessible without a using System; namespace declaration. For more information, see namespace (C# Reference).

Create test projects

Create two new xUnit Test Project (.NET Core) templates from the same command prompt using the dotnet new xunit command:

dotnet new xunit -n XUnit.Coverlet.Collector
dotnet new xunit -n XUnit.Coverlet.MSBuild

Both of the newly created xUnit test projects need to add a project reference of the Numbers class library. This is so that the test projects have access to the PrimeService for testing. From the command prompt, use the dotnet add command:

dotnet add XUnit.Coverlet.Collector\XUnit.Coverlet.Collector.csproj reference Numbers\Numbers.csproj
dotnet add XUnit.Coverlet.MSBuild\XUnit.Coverlet.MSBuild.csproj reference Numbers\Numbers.csproj

The MSBuild project is named appropriately, as it will depend on the coverlet.msbuild NuGet package. Add this package dependency by running the dotnet add package command:

cd XUnit.Coverlet.MSBuild && dotnet add package coverlet.msbuild && cd ..

The previous command changed directories effectively scoping to the MSBuild test project, then added the NuGet package. When that was done, it then changed directories, stepping up one level.

Open both of the UnitTest1.cs files, and replace their contents with the following snippet. Rename the UnitTest1.cs files to PrimeServiceTests.cs.

using System.Numbers;
using Xunit;

namespace XUnit.Coverlet
{
    public class PrimeServiceTests
    {
        readonly PrimeService _primeService;

        public PrimeServiceTests() => _primeService = new PrimeService();

        [
            Theory,
            InlineData(-1), InlineData(0), InlineData(1)
        ]
        public void IsPrime_ValuesLessThan2_ReturnFalse(int value) =>
            Assert.False(_primeService.IsPrime(value), $"{value} should not be prime");

        [
            Theory,
            InlineData(2), InlineData(3), InlineData(5), InlineData(7)
        ]
        public void IsPrime_PrimesLessThan10_ReturnTrue(int value) =>
            Assert.True(_primeService.IsPrime(value), $"{value} should be prime");

        [
            Theory,
            InlineData(4), InlineData(6), InlineData(8), InlineData(9)
        ]
        public void IsPrime_NonPrimesLessThan10_ReturnFalse(int value) =>
            Assert.False(_primeService.IsPrime(value), $"{value} should not be prime");
    }
}

Create a solution

From the command prompt, create a new solution to encapsulate the class library and the two test projects. Using the dotnet sln command:

dotnet new sln -n XUnit.Coverage

This will create a new solution file name XUnit.Coverage in the UnitTestingCodeCoverage directory. Add the projects to the root of the solution.

dotnet sln XUnit.Coverage.sln add **/*.csproj --in-root

Build the solution using the dotnet build command:

dotnet build

If the build is successful, you've created the three projects, appropriately referenced projects and packages, and updated the source code correctly. Well done!

Code coverage tooling

There are two types of code coverage tools:

  • DataCollectors: DataCollectors monitor test execution and collect information about test runs. They report the collected information in various output formats, such as XML and JSON. For more information, see your first DataCollector.
  • Report generators: Use data collected from test runs to generate reports, often as styled HTML.

In this section, the focus is on data collector tools. To use Coverlet for code coverage, an existing unit test project must have the appropriate package dependencies, or alternatively rely on .NET global tooling and the corresponding coverlet.console NuGet package.

Integrate with .NET test

The xUnit test project template already integrates with coverlet.collector by default. From the command prompt, change directories to the XUnit.Coverlet.Collector project, and run the dotnet test command:

cd XUnit.Coverlet.Collector && dotnet test --collect:"XPlat Code Coverage"

Note

The "XPlat Code Coverage" argument is a friendly name that corresponds to the data collectors from Coverlet. This name is required but is case insensitive.

As part of the dotnet test run, a resulting coverage.cobertura.xml file is output to the TestResults directory. The XML file contains the results. This is a cross-platform option that relies on the .NET CLI, and it is great for build systems where MSBuild is not available.

Below is the example coverage.cobertura.xml file.

<?xml version="1.0" encoding="utf-8"?>
<coverage line-rate="1" branch-rate="1" version="1.9" timestamp="1592248008"
          lines-covered="12" lines-valid="12" branches-covered="6" branches-valid="6">
  <sources>
    <source>C:\</source>
  </sources>
  <packages>
    <package name="Numbers" line-rate="1" branch-rate="1" complexity="6">
      <classes>
        <class name="Numbers.PrimeService" line-rate="1" branch-rate="1" complexity="6"
               filename="Numbers\PrimeService.cs">
          <methods>
            <method name="IsPrime" signature="(System.Int32)" line-rate="1"
                    branch-rate="1" complexity="6">
              <lines>
                <line number="8" hits="11" branch="False" />
                <line number="9" hits="11" branch="True" condition-coverage="100% (2/2)">
                  <conditions>
                    <condition number="7" type="jump" coverage="100%" />
                  </conditions>
                </line>
                <line number="10" hits="3" branch="False" />
                <line number="11" hits="3" branch="False" />
                <line number="14" hits="22" branch="True" condition-coverage="100% (2/2)">
                  <conditions>
                    <condition number="57" type="jump" coverage="100%" />
                  </conditions>
                </line>
                <line number="15" hits="7" branch="False" />
                <line number="16" hits="7" branch="True" condition-coverage="100% (2/2)">
                  <conditions>
                    <condition number="27" type="jump" coverage="100%" />
                  </conditions>
                </line>
                <line number="17" hits="4" branch="False" />
                <line number="18" hits="4" branch="False" />
                <line number="20" hits="3" branch="False" />
                <line number="21" hits="4" branch="False" />
                <line number="23" hits="11" branch="False" />
              </lines>
            </method>
          </methods>
          <lines>
            <line number="8" hits="11" branch="False" />
            <line number="9" hits="11" branch="True" condition-coverage="100% (2/2)">
              <conditions>
                <condition number="7" type="jump" coverage="100%" />
              </conditions>
            </line>
            <line number="10" hits="3" branch="False" />
            <line number="11" hits="3" branch="False" />
            <line number="14" hits="22" branch="True" condition-coverage="100% (2/2)">
              <conditions>
                <condition number="57" type="jump" coverage="100%" />
              </conditions>
            </line>
            <line number="15" hits="7" branch="False" />
            <line number="16" hits="7" branch="True" condition-coverage="100% (2/2)">
              <conditions>
                <condition number="27" type="jump" coverage="100%" />
              </conditions>
            </line>
            <line number="17" hits="4" branch="False" />
            <line number="18" hits="4" branch="False" />
            <line number="20" hits="3" branch="False" />
            <line number="21" hits="4" branch="False" />
            <line number="23" hits="11" branch="False" />
          </lines>
        </class>
      </classes>
    </package>
  </packages>
</coverage>

Tip

As an alternative, you could use the MSBuild package if your build system already makes use of MSBuild. From the command prompt, change directories to the XUnit.Coverlet.MSBuild project, and run the dotnet test command:

dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura

The resulting coverage.cobertura.xml file is output. You can follow MSBuild integration guide here

Generate reports

Now that you're able to collect data from unit test runs, you can generate reports using ReportGenerator. To install the ReportGenerator NuGet package as a .NET global tool, use the dotnet tool install command:

dotnet tool install -g dotnet-reportgenerator-globaltool

Run the tool and provide the desired options, given the output coverage.cobertura.xml file from the previous test run.

reportgenerator
-reports:"Path\To\TestProject\TestResults\{guid}\coverage.cobertura.xml"
-targetdir:"coveragereport"
-reporttypes:Html

After running this command, an HTML file represents the generated report.

Unit test-generated report

See also

Next Steps