Configure MSTest

MSTest, Microsoft Testing Framework, is a test framework for .NET applications. It allows you to write and execute tests, and provide test suites with integration to Visual Studio and Visual Studio Code Test Explorers, the .NET CLI, and many CI pipelines.

MSTest is a fully supported, open-source and a cross-platform test framework that works with all supported .NET targets (.NET Framework, .NET Core, .NET, UWP, WinUI, and so on) hosted on GitHub.

Runsettings

A .runsettings file can be used to configure how unit tests are being run. To learn more about the runsettings and the configurations related to the platform, you can check out VSTest runsettings documentation or MSTest runner runsettings documentation.

MSTest element

The following runsettings entries let you configure how MSTest behaves.

Configuration	Default	Values
AssemblyCleanupTimeout	None	Specify globally the timeout to apply on each instance of assembly cleanup method. [Timeout] attribute specified on the assembly cleanup method overrides the global timeout.
AssemblyInitializeTimeout	None	Specify globally the timeout to apply on each instance of assembly initialize method. [Timeout] attribute specified on the assembly initialize method overrides the global timeout.
AssemblyResolution	false	You can specify paths to extra assemblies when finding and running unit tests. For example, use these paths for dependency assemblies that aren't in the same directory as the test assembly. To specify a path, use a Directory Path element. Paths can include environment variables. <AssemblyResolution> <Directory path="D:\myfolder\bin\" includeSubDirectories="false"/> </AssemblyResolution> This feature is only applied when using a .NET Framework target.
CaptureTraceOutput	Result	Capture text from the Console.Write*, Trace.Write*, and Debug.Write* APIs and associate it with the current test. Starting with MSTest 4.4, use None, Result, or Live. Live also echoes Console, Trace, and TestContext.Write* output to the console while the test runs. The earlier Boolean values remain supported: true maps to Result, and false maps to None.
ClassCleanupLifecycle	EndOfClass	If you want the class cleanup to occur at the end of assembly, set it to EndOfAssembly. (No longer supported starting from MSTest v4 as EndOfClass is the default and only ClassCleanup behavior)
ClassCleanupTimeout	None	Specify globally the timeout to apply on each instance of class cleanup method. [Timeout] attribute specified on the class cleanup method overrides the global timeout.
ClassInitializeTimeout	None	Specify globally the timeout to apply on each instance of class initialize method. [Timeout] attribute specified on the class initialize method overrides the global timeout.
ConsiderFixturesAsSpecialTests	false	To display AssemblyInitialize, AssemblyCleanup, ClassInitialize, ClassCleanup as individual entries in Visual Studio and Visual Studio Code Test Explorer and .trx log, set this value to true
DeleteDeploymentDirectoryAfterTestRunIsComplete	true	To retain the deployment directory after a test run, set this value to false.
DeploymentEnabled	true	If you set the value to false, deployment items that you specify in your test method aren't copied to the deployment directory.
DeployTestSourceDependencies	true	A value indicating whether the test source references are to be deployed.
EnableBaseClassTestMethodsFromOtherAssemblies	true	A value indicating whether to enable discovery of test methods from base classes in a different assembly from the inheriting test class.
ForcedLegacyMode	false	In older versions of Visual Studio, the MSTest adapter was optimized to make it faster and more scalable. Some behavior, such as the order in which tests are run, might not be exactly as it was in previous editions of Visual Studio. Set the value to true to use the older test adapter. For example, you might use this setting if you have an app.config file specified for a unit test. We recommend that you consider refactoring your tests to allow you to use the newer adapter.
GlobalTestCleanupTimeout	TestCleanupTimeout	Starting with MSTest 4.4, specify the timeout for each global test cleanup method. When you omit this entry, MSTest uses TestCleanupTimeout. A [Timeout] attribute on the method overrides both values.
GlobalTestInitializeTimeout	TestInitializeTimeout	Starting with MSTest 4.4, specify the timeout for each global test initialize method. When you omit this entry, MSTest uses TestInitializeTimeout. A [Timeout] attribute on the method overrides both values.
LaunchDebuggerOnTestFailure	false	Starting with MSTest 4.2, when set to true, MSTest launches the debugger when a test fails.
MapInconclusiveToFailed	false	If a test completes with an inconclusive status, it's mapped to the skipped status in Test Explorer. If you want inconclusive tests to be shown as failed, set the value to true.
MapNotRunnableToFailed	true	A value indicating whether a not runnable result is mapped to failed test.
OrderTestsByNameInClass	false	If you want to run tests by test names both in Test Explorers and on the command line, set this value to true.
Parallelize		Used to set the parallelization settings: Workers: The number of threads/workers to be used for parallelization, which is by default the number of processors on the current machine. Scope: The scope of parallelization. You can set it to MethodLevel. By default, it's ClassLevel. <Parallelize><Workers>32</Workers><Scope>MethodLevel</Scope></Parallelize>
RandomizeTestOrder	false	Starting with MSTest 4.3, set this value to true to run tests in a random order, which helps surface hidden ordering dependencies between tests. This setting can't be combined with OrderTestsByNameInClass.
RandomTestOrderSeed		Starting with MSTest 4.3, when RandomizeTestOrder is true, set an integer seed to make the random order reproducible across runs. When unset, a new seed is used for each run.
SettingsFile		You can specify a test settings file to use with the MSTest adapter here. You can also specify a test settings file from the settings menu. If you specify this value, you must also set the ForcedLegacyMode to true. <ForcedLegacyMode>true</ForcedLegacyMode>
TestCleanupTimeout	None	Specify globally the timeout to apply on each instance of test cleanup method. [Timeout] attribute specified on the test cleanup method overrides the global timeout.
TestInitializeTimeout	None	Specify globally the timeout to apply on each instance of test initialize method. [Timeout] attribute specified on the test initialize method overrides the global timeout.
TestTimeout	None	Gets specified global test case timeout.
TreatClassAndAssemblyCleanupWarningsAsErrors	false	To see your failures in class cleanups as errors, set this value to true.
TreatDiscoveryWarningsAsErrors	false	To report test discovery warnings as errors, set this value to true.

Timeout values must be positive integers in milliseconds. To run without a timeout, omit the entry instead of setting it to 0. Global test fixture timeouts inherit the corresponding TestInitializeTimeout or TestCleanupTimeout value.

TestRunParameter element

<TestRunParameters>
    <Parameter name="webAppUrl" value="http://localhost" />
</TestRunParameters>

Test run parameters provide a way to define variables and values that are available to the tests at runtime. Access the parameters using the MSTest TestContext.Properties property:

private string _appUrl;
public TestContext TestContext { get; set; }

[TestMethod]
public void HomePageTest()
{
    string _appUrl = TestContext.Properties["webAppUrl"];
}

To use test run parameters, add a public TestContext property to your test class.

Example .runsettings file

The following XML shows the contents of a typical .runsettings file. Copy this code and edit it to suit your needs.

Each element of the file is optional because it has a default value.

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>

  <!-- Parameters used by tests at runtime -->
  <TestRunParameters>
    <Parameter name="webAppUrl" value="http://localhost" />
    <Parameter name="webAppUserName" value="Admin" />
    <Parameter name="webAppPassword" value="Password" />
  </TestRunParameters>

  <!-- MSTest -->
  <MSTest>
    <MapInconclusiveToFailed>True</MapInconclusiveToFailed>
    <CaptureTraceOutput>false</CaptureTraceOutput>
    <DeleteDeploymentDirectoryAfterTestRunIsComplete>False</DeleteDeploymentDirectoryAfterTestRunIsComplete>
    <DeploymentEnabled>False</DeploymentEnabled>
    <ConsiderFixturesAsSpecialTests>False</ConsiderFixturesAsSpecialTests>
    <AssemblyResolution>
      <Directory path="D:\myfolder\bin\" includeSubDirectories="false"/>
    </AssemblyResolution>
  </MSTest>

</RunSettings>

testconfig.json

When running your tests with MSTest, you can use a testconfig.json file to configure the behavior of the test runner. The testconfig.json file is a JSON file that contains the configuration settings for the test runner. The file is used to configure the test runner and the test execution environment. For more information, refer to MTP testconfig.json documentation.

Starting with MSTest 3.7, you can also configure MSTest runs in the same configuration file. The following sections describe the settings that you can use in the testconfig.json file.

Starting with MSTest 4.3.3, .NET Framework runs also accept comments and trailing commas in testconfig.json.

MSTest element

MSTest settings are grouped by functionality that are described in the sections that follow.

Entry	Default	Description
enableBaseClassTestMethodsFromOtherAssemblies	true	A value indicating whether to enable discovery of test methods from base classes in a different assembly from the inheriting test class.
classCleanupLifecycle	EndOfAssembly	If you want the class cleanup to occur at the end of the class, set it to EndOfClass.

assemblyResolution settings

All the settings in this section belong to the assemblyResolution element.

Entry	Default	Description
paths	None	You can specify paths to extra assemblies when finding and running unit tests. For example, use these paths for dependency assemblies that aren't in the same directory as the test assembly. You can specify a path in the shape { "path": "...", "includeSubDirectories": "true/false" }.

Example:

{
  "mstest": {
    "assemblyResolution": {
        { "path": "...", "includeSubDirectories": "true/false" }
    }
  }
}

deployment settings

All the settings in this section belong to the deployment element.

Entry	Default	Description
deleteDeploymentDirectoryAfterTestRunIsComplete	true	To retain the deployment directory after a test run, set this value to false.
deployTestSourceDependencies	true	Indicates whether the test source references are to be deployed.
enabled	true	If you set the value to false, deployment items that you specify in your test method aren't copied to the deployment directory.

Example:

{
  "mstest": {
    "deployment": {
        "deleteDeploymentDirectoryAfterTestRunIsComplete": true,
        "deployTestSourceDependencies": true,
        "enabled": true
    }
  }
}

output settings

All the settings in this section belong to the output element.

Entry	Default	Description
captureTrace	Result	Capture Console, Trace, and Debug output and associate it with the current test. Starting with MSTest 4.4, use None, Result, or Live. Live also echoes output, including TestContext.Write* messages, while the test runs. The Boolean values remain supported: true maps to Result, and false maps to None.

Example:

{
  "mstest": {
    "output": {
        "captureTrace": false
    }
  }
}

parallelism settings

All the settings in this section belong to the parallelism element.

Entry	Default	Description
enabled	false	Enable test parallelization.
scope	class	The scope of parallelization. You can set it to method. The default, class, corresponds to running all tests of a given class sequentially but multiple classes in parallel.
workers	0	The number of threads/workers to be used for parallelization. The default value maps to the number of processors on the current machine.

Example:

{
  "mstest": {
    "parallelism": {
        "enabled": true,
        "scope": "method",
        "workers": 32
    }
  }
}

execution settings

All the settings in this section belong to the execution element.

Entry	Default	Description
considerEmptyDataSourceAsInconclusive	false	When set to true, an empty data source is considered as inconclusive.
considerFixturesAsSpecialTests	false	To display AssemblyInitialize, AssemblyCleanup, ClassInitialize, ClassCleanup as individual entries in Visual Studio and Visual Studio Code Test Explorer and .trx log, set this value to true.
dependencies		Starting with MSTest 4.4, declare test dependency chains and nodes. This setting is available only with Microsoft.Testing.Platform. For more information, see Test dependencies.
mapInconclusiveToFailed	false	If a test completes with an inconclusive status, it's mapped to the skipped status in Test Explorer. If you want inconclusive tests to be shown as failed, set the value to true.
launchDebuggerOnTestFailure	false	Starting with MSTest 4.2, when set to true, MSTest launches the debugger when a test fails.
mapNotRunnableToFailed	true	A value indicating whether a not runnable result is mapped to failed test.
orderTestsByNameInClass	false	Run tests in alphabetical order within each class. Starting with MSTest 4.3, use mstest.execution.orderTestsByNameInClass. The earlier mstest.orderTestsByNameInClass key still works but produces a deprecation warning.
randomizeTestOrder	false	Starting with MSTest 4.3, set this value to true to run tests in a random order, which helps surface hidden ordering dependencies between tests. This setting can't be combined with orderTestsByNameInClass.
randomTestOrderSeed		Starting with MSTest 4.3, when randomizeTestOrder is true, set an integer seed to make the random order reproducible across runs. When unset, a new seed is used for each run.
treatClassAndAssemblyCleanupWarningsAsErrors	false	To see your failures in class cleanups as errors, set this value to true.
treatDiscoveryWarningsAsErrors	false	To report test discovery warnings as errors, set this value to true.

Example:

{
  "mstest": {
    "execution": {
        "considerEmptyDataSourceAsInconclusive": false,
        "considerFixturesAsSpecialTests": false,
        "mapInconclusiveToFailed": true,
        "mapNotRunnableToFailed": true,
        "treatClassAndAssemblyCleanupWarningsAsErrors": false,
        "treatDiscoveryWarningsAsErrors": false
    }
  }
}

timeout settings

All the settings in this section belong to the timeout element.

Entry	Default	Description
assemblyCleanup	None	Specify globally the timeout to apply on each instance of assembly cleanup method.
assemblyInitialize	None	Specify globally the timeout to apply on each instance of assembly initialize method.
classCleanup	None	Specify globally the timeout to apply on each instance of class cleanup method.
classInitialize	None	Specify globally the timeout to apply on each instance of class initialize method.
globalTestCleanup	testCleanup	Starting with MSTest 4.4, specify the timeout for each global test cleanup method. When you omit this entry, MSTest uses testCleanup.
globalTestInitialize	testInitialize	Starting with MSTest 4.4, specify the timeout for each global test initialize method. When you omit this entry, MSTest uses testInitialize.
test	None	Specify globally the test timeout.
testCleanup	None	Specify globally the timeout to apply on each instance of test cleanup method.
testInitialize	None	Specify globally the timeout to apply on each instance of test initialize method.
useCooperativeCancellation	false	When set to true, in case of timeout, MSTest will only trigger cancellation of the CancellationToken but will not stop observing the method. This behavior is more performant but relies on the user to correctly flow the token through all paths.

Note

Timeout values must be positive integers in milliseconds. To run without a timeout, omit the entry instead of setting it to 0. Global test fixture timeouts inherit the corresponding testInitialize or testCleanup value, so omit both entries when you don't want a timeout on a global fixture. A [Timeout] attribute on a method overrides the configured timeout.

Example:

{
  "mstest": {
    "timeout": { "globalTestInitialize": 30000, "globalTestCleanup": 30000 }
  }
}

Example testconfig.json file

The following JSON shows the contents of a typical .testconfig.json file. Copy this code and edit it to suit your needs.

Each element of the file is optional because it has a default value.

{
  "platformOptions": {
    "resultDirectory": "./TestResults"
  },
  "mstest": {
    "execution": {
        "mapInconclusiveToFailed": true,
        "disableAppDomain": true,
        "considerFixturesAsSpecialTests": false
    },
    "parallelism": {
        "enabled": true,
        "scope": "method"
    },
    "output": {
        "captureTrace": false
    }
  }
}

MSBuild properties

Starting with MSTest 4.3, opt in to assembly-level parallelization from your project file or Directory.Build.props without authoring an [assembly: Parallelize] attribute. These properties emit the corresponding assembly attribute during build, so they require GenerateAssemblyInfo to be true (the default for SDK-style projects).

Property	Default	Description
MSTestParallelizeScope		The parallelization scope. Set it to MethodLevel or ClassLevel to emit [assembly: Parallelize(Scope = ExecutionScope.MethodLevel)] (or ExecutionScope.ClassLevel), or to None to emit [assembly: DoNotParallelize].
MSTestParallelizeWorkers		The maximum number of worker threads, emitted as the Workers value of [assembly: Parallelize]. A value of 0 maps to the number of processors on the current machine. This property can't be set when MSTestParallelizeScope is None.

MSTest validates both properties during the build. Invalid scope values, non-integer worker counts, and a worker count combined with a None scope fail the build. Don't also declare [assembly: Parallelize] or [assembly: DoNotParallelize] in source, because the generated attribute would duplicate it. When GenerateAssemblyInfo is false, declare the attribute in source instead.

The following example enables method-level parallelization with four workers for every test project that imports the Directory.Build.props file:

<Project>
  <PropertyGroup>
    <MSTestParallelizeScope>MethodLevel</MSTestParallelizeScope>
    <MSTestParallelizeWorkers>4</MSTestParallelizeWorkers>
  </PropertyGroup>
</Project>