Running Unit Tests with Test Explorer

  • Article

Visual Studio Test Explorer helps you integrate unit testing in your development process. You can run tests from multiple test projects in a solution and from test classes that are part of the production code projects. Test projects can use different unit test frameworks. In this topic, you can learn about:

In this section

Unit test frameworks and test projects

Running tests in Test Explorer

  • Running tests

  • Running tests after every build

Viewing test results

  • Viewing test details

  • Viewing the source code of a test method

Grouping and filtering the test list

  • Grouping the test list

  • Grouping by traits

  • Searching and filtering the test list

Debugging unit tests

Analyzing unit test code coverage

Diagnosing test method performance issues

External resources

  • Guidance

Unit test frameworks and test projects

Visual Studio includes the Microsoft unit testing frameworks for managed and for native code. However, Test Explorer can also run any unit test framework that has implemented a Test Explorer adapter. For more information about installing third-party unit test frameworks, see How to: Install Third-Party Unit Test Frameworks

Test Explorer can run tests from multiple test projects in a solution and from test classes that are part of the production code projects. Test projects can use different unit test frameworks. When the code under test is written for the .NET framework, the test project can be written in any .NET language, regardless of the language of the target code. Native C/C++ code projects must be tested by using a C++ unit test framework.

In this section

Running tests in Test Explorer

When you build the test project, the tests appear in Test Explorer. If Test Explorer is not visible, choose Test on the Visual Studio menu, choose Windows, and then choose Test Explorer.

Unit Test Explorer

As you run, write, and rerun your tests, Test Explorer displays the results in default groups of Failed Tests, Passed Tests, Skipped Tests and Not Run Tests. You can change the way Test Explorer groups your tests.

You can perform much of the work of finding, organizing and running tests from the Test Explorer toolbar.

Run tests from the Test Explorer toolbar

Running tests

You can run all the tests in the solution, all the tests in a group, or a set of tests that you select. Do one of the following:

  • To run all the tests in a solution, choose Run All.

  • To run all the tests in a default group, choose Run... and then choose the group on the menu.

  • Select the individual tests that you want to run, open the shortcut menu for a selected test and then choose Run Selected Tests.

The pass/fail bar at the top of the Test Explorer window is animated as the tests run. At the conclusion of the test run, the pass/fail bar turns green if all tests passed or turns red if any test failed.

Running tests after every build

Warning

Running unit tests after every build is supported only in Visual Studio Ultimate.

Run after build

To run your unit tests after each local build, choose Test on the standard menu, choose Run Tests After Build on the Test Explorer toolbar.

In this section

Viewing test results

As you run, write, and rerun your tests, Test Explorer displays the results in groups of Failed Tests, Passed Tests, Skipped Tests and Not Run Tests. The details pane at the bottom of Test Explorer displays a summary of the test run.

Viewing test details

To view the details of an individual test, select the test.

The test details pane displays the following information:

  • The source file name and the line number of the test method.

  • The status of the test.

  • The elapsed time that the test method took to run.

If the test fails, the details pane also displays:

  • The message returned by the unit test framework for the test.

  • The stack trace at the time the test failed.

Viewing the source code of a test method

To display the source code for a test method in the Visual Studio editor, select the test and then choose Open Test on the shortcut menu (Keyboard: F12).

In this section

Grouping and filtering the test list

Test Explorer lets you group your tests into predefined categories. Most unit test frameworks that run in Test Explorer let you define your own categories and category/value pairs to group your tests. You can also filter the list of tests by matching strings against test properties.

Grouping the test list

To change the way that tests are organized, choose the down arrow next to the Group By button Test Explorer group button and select a new grouping criteria.

Test Explorer groups

Group

Description

Duration

Groups test by execution time: Fast, Medium, and Slow.

Outcome

Groups tests by execution results: Failed Tests, Skipped Tests, Passed Tests.

Traits

Groups test bycategory/value pairs that you define. The syntax to specify trait categories and values is defined by the unit test framework.

Project

Groups test by the name of the projects.

Grouping by traits

A trait is usually a category name / value pair, but it can also be a single category. Traits can be assigned to methods that are identified as a test method by the unit test framework. A unit test framework can define trait categories. You can add values to the trait categories to define your own category name / value pairs. The syntax to specify trait categories and values is defined by the unit test framework.

For example, in the Microsoft unit test framework for managed apps, you define a trait name / value pair by using the TestPropertyAttribute. The Microsoft unit test framework also contains these pre-defined traits:

Trait

Description

OwnerAttribute

The Owner category is defined by the unit test framework and requires you to provide a string value of the owner.

PriorityAttribute

The Priority category is defined by the unit test framework and requires you to provide an integer value of the priority.

TestCategoryAttribute

The TestCategory attribute enables you to provide a category without a value. A category defined by the TestCategory attribute can also be the category of a TestProperty attribute.

TestPropertyAttribute

The TestProperty attribute enables you to define trait category/value pair.

Traits in the Microsoft Unit Testing Framework for C++

To define a trait, use the TEST_METHOD_ATTRIBUTE macro. For example, to define a trait named TEST_MY_TRAIT:

#define TEST_MY_TRAIT(traitValue) TEST_METHOD_ATTRIBUTE(L"MyTrait", traitValue)

To use the defined trait in your unit tests:

BEGIN_TEST_METHOD_ATTRIBUTE(Method1)
    TEST_OWNER(L"OwnerName")
    TEST_PRIORITY(1)
    TEST_MY_TRAIT(L"thisTraitValue")
END_TEST_METHOD_ATTRIBUTE()

TEST_METHOD(Method1)
{   
    Logger::WriteMessage("In Method1");
    Assert::AreEqual(0, 0);
}

C++ trait attribute macros

Macro

Description

TEST_METHOD_ATTRIBUTE(attributeName, attributeValue)

Use the TEST_METHOD_ATTRIBUTE macro to define a trait.

TEST_OWNER(ownerAlias)

Use the predefined Owner trait to specify an owner of the test method.

TEST_PRIORITY(priority)

Use the predefined Priority trait to assign relative priorities to your test methods.

In this section

Searching and filtering the test list

You can use Test Explorer filters to limit the test methods in your projects that you view and run.

When you type a string in in the Test Explorer search box and press ENTER, the test list is filtered to display only those tests whose fully qualified names contain the string.

To filter by a different criteria:

  1. Open the drop-down list to the right of the search box.

  2. Choose a new criteria.

  3. Enter the filter value between the quotation marks.

Search filter categories

Note

Searches are case insensitive and match the specified string to any part of the criteria value.

Qualifier

Description

Trait

Searches both trait category and value for matches. The syntax to specify trait categories and values are defined by the unit test framework.

Project

Searches the test project names for matches.

Error Message

Searches the user-defined error messages returned by failed asserts for matches.

File Path

Searches the fully qualified file name of test source files for matches.

Fully Qualified Name

Searches the fully qualified file name of test namespaces, classes, and methods for matches.

Output

Searches the user-defined error messages that are written to standard output (stdout) or standard error (stderr). The syntax to specify output messages are defined by the unit test framework.

Outcome

Searches the Test Explorer category names for matches: Failed Tests, Skipped Tests, Passed Tests.

In this section

Debugging unit tests

You can use Test Explorer to start a debugging session for your tests. Stepping through your code with the Visual Studio debugger seamlessly takes you back and forth between the unit tests and the project under test. To start debugging:

  1. In the Visual Studio editor, set a breakpoint in one or more test methods that you want to debug.

    Note

    Because test methods can run in any order, set breakpoints in all the test methods that you want to debug.

  2. In Test Explorer, select the test methods and then choose Debug Selected Tests on the shortcut menu.

For more information, about the debugger, see Debugging in Visual Studio.

In this section

Analyzing unit test code coverage

Note

Unit test code coverage is available only in Visual Studio Ultimate and Visual Studio Premium.

You can determine the amount of your product code that is actually being tested by your unit tests by using the Visual Studio code coverage tool. You can run code coverage on selected tests or on all tests in a solution.

To run code coverage for test methods in a solution:

  1. Choose Tests on the Visual Studio menu and then choose Analyze code coverage.

  2. Choose one of the following commands from the sub-menu:

    • Selected tests runs the test methods that you have selected in Test Explorer.

    • All tests runs all the test methods in the solution.

The Code Coverage Results window displays the percentage of the blocks of product code that were exercised by line, function, class, namespace and module.

For more information, see Using Code Coverage to Determine How Much Code is being Tested.

In this section

Diagnosing test method performance issues

To diagnose why a test method is taking too much time, select the method in Test Explorer and then choose Profile on the shortcut menu. See Analyzing Application Performance by Using Profiling Tools

In this section

External resources

Guidance

Testing for Continuous Delivery with Visual Studio 2012 – Chapter 2: Unit Testing: Testing the Inside

See Also

Concepts

Verifying Code by Using Unit Tests

How to: Run a Unit Test as a 64-bit Process