Live Unit Testing frequently asked questions

Live Unit Testing works with the three popular unit testing frameworks listed in the table that follows. The minimum supported version of their adapters and frameworks is also listed in the table. The unit testing frameworks are all available from NuGet.org.

If you have older MSTest based test projects that reference Microsoft.VisualStudio.QualityTools.UnitTestFramework and you don’t wish to move to the newer MSTest NuGet packages, upgrade to Visual Studio 2019 or Visual Studio 2017.

In some cases, you may need to explicitly restore the NuGet packages referenced by the projects in the solution in order for Live Unit Testing to work. You can restore the packages either by doing an explicit build of the solution (select Build > Rebuild Solution from the top-level Visual Studio menu), or by right-clicking on the solution and selecting Restore NuGet Packages before enabling Living Unit Testing.

Yes. Live Unit Testing works with .NET Core and the .NET Framework.

The Output window (when the Live Unit Testing drop-down is selected) should tell you why Live Unit Testing is not working. Live Unit Testing may not work for one of the following reasons:

• If NuGet packages referenced by the projects in the solution have not been restored, Live Unit Testing will not work. Doing an explicit build of the solution or restoring NuGet packages in the solution before turning on Live Unit Testing should resolve this issue.

• If you are using MSTest-based tests in your projects, make sure that you remove the reference to Microsoft.VisualStudio.QualityTools.UnitTestFramework, and add references to the latest MSTest NuGet packages, MSTest.TestAdapter (a minimum version of 1.1.11 is required) and MSTest.TestFramework (a minimum version of 1.1.11 is required). For more information, see the "Supported test frameworks" section of the Use Live Unit Testing in Visual Studio article.

• At least one project in your solution should have either a NuGet reference or direct reference to the xUnit, NUnit, or MSTest test framework. This project should also reference a corresponding Visual Studio test adapters NuGet package. The Visual Studio test adapter can also be referenced through a .runsettings file. The .runsettings file must have an entry like the following example:

<RunSettings>
    <RunConfiguration>
          <TestAdaptersPaths>path-to-your-test-adapter</TestAdaptersPaths>
     </RunConfiguration>
</RunSettings>

• If multiple projects in the solution reference the NuGet test adapter package, each of them must be upgraded to the supported version.

• Make sure the MSBuild .props file imported from the test adapter package is correctly updated as well. Check the NuGet package version/path of the import, which can usually be found near the top of the project file, like the following:

    <Import Project="..\packages\xunit.runner.visualstudio.2.2.0\build\net20\xunit.runner.visualstudio.props" Condition="Exists('..\packages\xunit.runner.visualstudio.2.2.0\build\net20\xunit.runner.visualstudio.props')" />
  

If your solution requires custom steps to build for instrumentation (Live Unit Testing) that are not required for the "regular" non-instrumented build, then you can add code to your project or .targets files that checks for the BuildingForLiveUnitTesting property and performs custom pre/post build steps. You can also choose to remove certain build steps (like publishing or generating packages) or to add build steps (like copying prerequisites) to a Live Unit Testing build based on this project property. Customizing your build based on this property does not alter your regular build in any way, and only impacts Live Unit Testing builds.

For example, there may be a target that produces NuGet packages during a regular build. You probably do not want NuGet packages to be generated after every edit you make. So you can disable that target in the Live Unit Testing build by doing something like the following:  

<Target Name="GenerateNuGetPackages" BeforeTargets="AfterBuild" Condition="'$(BuildingForLiveUnitTesting)' != 'true'">
    <Exec Command='"$(MSBuildThisFileDirectory)..\tools\GenPac" '/>
</Target>

You can get this error if the build process for your solution has custom logic that specifies where binaries should be generated. By default the location of your binaries depends on <OutputPath>, <OutDir> or <IntermediateOutputPath> as well as <BaseOutputPath> or <BaseIntermediateOutputPath>.

Live Unit Testing overrides those variables to ensure that build artifacts are dropped to a Live Unit Testing artifacts folder and will fail if your build process also overrides these variables.

There are two main approaches to make Live Unit Testing build successfully. For easier build configurations, you can base your output paths on <BaseIntermediateOutputPath>. For more complex configurations you can base your output paths on <LiveUnitTestingBuildRootPath>.

During build, Live Unit Testing automatically overrides the <BaseOutputPath>/<BaseIntermediateOutputPath> variables to target the Live Unit Testing artifacts folder.

For example, if your build overrides the <OutputPath> as shown below:

<Project>
  <PropertyGroup>
    <OutputPath>$(SolutionDir)Artifacts\$(Configuration)\bin\$(MSBuildProjectName)</OutputPath>
  </PropertyGroup>
</Project>

then you can replace it with the following XML:

<Project>
  <PropertyGroup>
    <BaseOutputPath Condition="'$(BaseOutputPath)' == ''">$(SolutionDir)Artifacts\$(Configuration)\bin\$(MSBuildProjectName)\</BaseOutputPath>
    <OutputPath Condition="'$(OutputPath)' == ''">$(BaseOutputPath)</OutputPath>
  </PropertyGroup>
</Project>

This ensures that <OutputPath> lies within the <BaseOutputPath> folder.

Do not override <OutDir> directly in your build process; override <OutputPath> instead to drop build artifacts to a specific location.

During a Live Unit Testing build, the <LiveUnitTestingBuildRootPath> property is set to the location of Live Unit Testing artifacts folder.

For example, assume that your project has the structure shown here.

.vs\...\lut\0\b
artifacts\{binlog,obj,bin,nupkg,testresults,packages}
src\{proj1,proj2,proj3}
tests\{testproj1,testproj2}
Solution.sln

During the Live Unit Testing build, the <LiveUnitTestingBuildRootPath> property is set to the full path of .vs\...\lut\0\b. If the project defines <ArtifactsRoot> property that maps to the solution dir, you can update the MSBuild project as follows:

<Project>
    <PropertyGroup Condition="'$(LiveUnitTestingBuildRootPath)' == ''">
        <SolutionDir>$([MSBuild]::GetDirectoryNameOfFileAbove(`$(MSBuildProjectDirectory)`, `YOUR_SOLUTION_NAME.sln`))\</SolutionDir>

        <ArtifactsRoot>Artifacts\</ArtifactsRoot>
        <ArtifactsRoot Condition="'$(LiveUnitTestingBuildRootPath)' != ''">$(LiveUnitTestingBuildRootPath)</ArtifactsRoot>
    </PropertyGroup>

    <PropertyGroup>
        <BinLogPath>$(ArtifactsRoot)\BinLog</BinLogPath>
        <ObjPath>$(ArtifactsRoot)\Obj</ObjPath>
        <BinPath>$(ArtifactsRoot)\Bin</BinPath>
        <NupkgPath>$(ArtifactsRoot)\Nupkg</NupkgPath>
        <TestResultsPath>$(ArtifactsRoot)\TestResults</TestResultsPath>

        <!-- Note: Given that a build doesn't generate packages, the path should be relative to the solution dir, rather than artifacts root, which will change during a Live Unit Testing build. -->
        <PackagesPath>$(SolutionDir)\artifacts\packages</PackagesPath>
    </PropertyGroup>
</Project>

Set the LiveUnitTesting_BuildRoot user-level environment variable to the path where you want the Live Unit Testing build artifacts to be dropped. 

There are several differences:

• Running or debugging tests from the Test Explorer window runs regular binaries, whereas Live Unit Testing runs instrumented binaries. If you want to debug instrumented binaries, adding a Debugger.Launch method call in your test method causes the debugger to launch whenever that method is executed (including when it is executed by Live Unit Testing), and you can then attach and debug the instrumented binary. However, our hope is that instrumentation is transparent to you for most user scenarios, and that you do not need to debug instrumented binaries.

• Live Unit Testing does not create a new application domain to run tests, but tests run from the Test Explorer window do create a new application domain.

• Live Unit Testing runs tests in each test assembly sequentially. In Test Explorer, you can choose to run multiple tests in parallel.

• Test Explorer runs tests in a single-threaded apartment (STA) by default, whereas Live Unit Testing runs tests in a multi-threaded apartment (MTA). To run MSTest tests in STA in Live Unit Testing, decorate the test method or the containing class with the <STATestMethod> or <STATestClass> attribute that can be found in the MSTest.STAExtensions 1.0.3-beta NuGet package. For NUnit, decorate the test method with the <RequiresThread(ApartmentState.STA)> attribute, and for xUnit, with the <STAFact> attribute.

See the "Include and exclude test projects and test methods" section of the Use Live Unit Testing in Visual Studio article for the user-specific setting. Including or excluding tests is useful when you want to run a specific set of tests for a particular edit session or to persist your own personal preferences.

For solution-specific settings, you can apply the System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute attribute programmatically to exclude methods, properties, classes, or structures from being instrumented by Live Unit Testing. Additionally, you can also set the <ExcludeFromCodeCoverage> property to true in your project file to exclude the whole project from being instrumented. Live Unit Testing will still run the tests that have not been instrumented, but their coverage will not be visualized.

You can also check whether Microsoft.CodeAnalysis.LiveUnitTesting.Runtime is loaded in the current application domain and disable tests based on why. For example, you can do something like the following with xUnit:

[ExcludeFromCodeCoverage]
public class SkipLiveFactAttribute : FactAttribute
{
   private static bool s_lutRuntimeLoaded = AppDomain.CurrentDomain.GetAssemblies().Any(a => a.GetName().Name ==
                                            "Microsoft.CodeAnalysis.LiveUnitTesting.Runtime");
   public override string Skip => s_lutRuntimeLoaded ? "Test excluded from Live Unit Testing" : "";
}

public class Class1
{
   [SkipLiveFact]
   public void F()
   {
      Assert.True(true);
   }
}

Your solution can build even if you're not making edits if the build process generates source code that's part of the solution itself, and your build target files don't have appropriate inputs and outputs specified. Targets should be given a list of inputs and outputs so that MSBuild can perform the appropriate up-to-date checks and determine whether a new build is required.

Live Unit Testing starts a build whenever it detects that source files have changed. Because the build of your solution generates source files, Live Unit Testing gets into an infinite build loop. If, however, the inputs and outputs of the target are checked when Live Unit Testing starts the second build (after detecting the newly generated source files from the previous build), it breaks out of the build loop because the inputs and outputs checks indicate that everything is up-to-date.

You might not see icons in the editor if the assemblies that Live Unit Testing is operating on aren't instrumented for any reason. For example, Live Unit Testing is not compatible with projects that set <UseHostCompilerIfAvailable>false</UseHostCompilerIfAvailable>. In this case, your build process needs to be updated to either remove this setting or to change it to true for Live Unit Testing to work. 

You can do several things to collect more detailed logs:

• Go to Tools > Options > Live Unit Testing and change the logging option to Verbose. Verbose logging causes more detailed logs to be shown in the Output window.

• Set the LiveUnitTesting_BuildLog user environment variable to the name of the file you want to use to capture the MSBuild log. Detailed MSBuild log messages from Live Unit Testing builds can then be retrieved from that file.

• Set the LiveUnitTesting_TestPlatformLog user environment variable to 1 to capture the Test Platform log. Detailed Test Platform log messages from Live Unit Testing runs can then be retrieved from [Solution Root]\.vs\[Solution Name]\log\[VisualStudio Process ID].

• Create a user-level environment variable named VS_UTE_DIAGNOSTICS and set it to 1 (or any value) and restart Visual Studio. Now you should see lots of logging in the Output - Tests tab in Visual Studio.

See also

• Live Unit Testing