Live Unit Testing frequently asked questions

Live Unit Testing works with the three popular unit testing frameworks listed in the table that follows. The minimum supported version of their adapters and frameworks is also listed in the table. The unit testing frameworks are all available from NuGet.org.

If you have older MSTest based test projects that reference Microsoft.VisualStudio.QualityTools.UnitTestFramework and you don’t wish to move to the newer MSTest NuGet packages, upgrade to Visual Studio 2019 or Visual Studio 2017.

Yes. Live Unit Testing works with .NET Core and the .NET Framework.

The Output window (when the Live Unit Testing drop-down is selected) should tell you why Live Unit Testing isn't working. Live Unit Testing may not work for one of the following reasons:

• If NuGet packages referenced by the projects in the solution haven't been restored, Live Unit Testing won't work. Performing an explicit build of the solution or restoring NuGet packages in the solution before turning on Live Unit Testing should resolve this issue.

• If you are using MSTest-based tests in your projects, make sure that you remove the reference to Microsoft.VisualStudio.QualityTools.UnitTestFramework, and add references to the latest MSTest NuGet packages, MSTest.TestAdapter (a minimum version of 1.1.11 is required) and MSTest.TestFramework (a minimum version of 1.1.11 is required). For more information, see the "Supported test frameworks" section of the Use Live Unit Testing in Visual Studio article.

• At least one project in your solution should have either a NuGet reference or direct reference to the xUnit, NUnit, or MSTest test framework. This project should also reference a corresponding Visual Studio test adapters NuGet package.

The build errors get reported to the Output window when the Live Unit Testing drop-down is selected. There are a few common issues caused by incorrect configuration in the setup wizard that can cause build issues in Live Unit Testing.

• If the Workspace Root property is too long, it’s possible that the build will fail due to exceptions indicating that the path is too long.

• If the Repository Root property doesn't point towards the repository root, the workspace will be populated with the wrong set of files.

• For git repositories, the Exclude files property usually avoids copying the files specified in the gitignore file. However, it’s possible to check in files to the git repository that are ignored, or it’s possible to run tools that auto generate files, but these are not generated during the build. In these cases, the "<Custom>" option should be chosen and a custom set of rules that only list the artifacts folders should be listed.

Besides the issues described previously, the following project configurations that might not build correctly.

• If project dependencies are specified as a global solution configuration and not as ProjectReferences for each project, Live Unit Testing might end up building the incorrect set of projects. To fix this, add explicit references between projects.

• Until a Live Unit Testing playlist is chosen, Live Unit Testing will not build any projects. To fix this, include some tests in the Live Unit Testing playlist.

• If you're using MSTest-based tests in your projects, make sure that you remove the reference to Microsoft.VisualStudio.QualityTools.UnitTestFramework, and add references to the latest MSTest NuGet packages, MSTest.TestAdapter (a minimum version of 1.1.11 is required) and MSTest.TestFramework (a minimum version of 1.1.11 is required). For more information, see Supported test frameworks.

• At least one project in your solution should have either a NuGet reference or direct reference to the xUnit, NUnit, or MSTest test framework. This project should also reference a corresponding Visual Studio test adapters NuGet package. The Visual Studio test adapter can also be referenced through a .runsettings file. The .runsettings file must have an entry like the following example:

<RunSettings>
    <RunConfiguration>
          <TestAdaptersPaths>path-to-your-test-adapter</TestAdaptersPaths>
    </RunConfiguration>
</RunSettings>

• A common issue is that not all of the files are copied to the test folder. You might have to add some Live Unit Testing Test Dependency items to the csproj files.

• Another issue is timeouts. Since Live Unit Testing runs tests indefinitely, it automatically aborts a run if the tests run for too long. You may increase the timeout in the project’s Wizard.

• If multiple projects in the solution reference the NuGet test adapter package, each of them must be upgraded to the supported version.

• Make sure the MSBuild .props file imported from the test adapter package is correctly updated as well. Check the NuGet package version/path of the import, which can usually be found near the top of the project file, like the following:

    <Import Project="..\packages\xunit.runner.visualstudio.2.2.0\build\net20\xunit.runner.visualstudio.props" Condition="Exists('..\packages\xunit.runner.visualstudio.2.2.0\build\net20\xunit.runner.visualstudio.props')" />
  

If your solution requires custom steps to build for instrumentation (Live Unit Testing) that aren't required for the "regular" non-instrumented build, then you can add code to your project or .targets files that checks for the BuildingForLiveUnitTesting property and performs custom pre/post build steps. You can also choose to remove certain build steps (like publishing or generating packages) or to add build steps (like copying prerequisites) to a Live Unit Testing build based on this project property. Customizing your build based on this property doesn't alter your regular build in any way, and only impacts Live Unit Testing builds.

For example, there may be a target that produces NuGet packages during a regular build. You probably do not want NuGet packages to be generated after every edit you make. So you can disable that target in the Live Unit Testing build by doing something like the following:  

<Target Name="GenerateNuGetPackages" BeforeTargets="AfterBuild" Condition="'$(BuildingForLiveUnitTesting)' != 'true'">
    <Exec Command='"$(MSBuildThisFileDirectory)..\tools\GenPac" '/>
</Target>

There are several differences:

• Running or debugging tests from the Test Explorer window runs regular binaries, whereas Live Unit Testing runs instrumented binaries. If you want to debug instrumented binaries, adding a Debugger.Launch method call in your test method causes the debugger to launch whenever that method is executed (including when it is executed by Live Unit Testing), and you can then attach and debug the instrumented binary. However, we hope that the instrumentation is transparent to you for most user scenarios, and that you don't need to debug instrumented binaries.

• Live Unit Testing doesn't create a new application domain to run tests, but tests run from the Test Explorer window do create a new application domain.

• Live Unit Testing runs tests in each test assembly sequentially. In Test Explorer, you can choose to run multiple tests in parallel.

• Test Explorer runs tests in a single-threaded apartment (STA) by default, whereas Live Unit Testing runs tests in a multi-threaded apartment (MTA). To run MSTest tests in STA in Live Unit Testing, decorate the test method or the containing class with the <STATestMethod> or <STATestClass> attribute that can be found in the MSTest.STAExtensions 1.0.3-beta NuGet package. For NUnit, decorate the test method with the <RequiresThread(ApartmentState.STA)> attribute, and for xUnit, with the <STAFact> attribute.

See the "Include and exclude test projects and test methods" section of the Use Live Unit Testing in Visual Studio article for the user-specific setting. Including or excluding tests is useful when you want to run a specific set of tests for a particular edit session or to persist your own personal preferences.

For solution-specific settings, you can apply the System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute attribute programmatically to exclude methods, properties, classes, or structures from being instrumented by Live Unit Testing. Additionally, you can also set the <ExcludeFromCodeCoverage> property to true in your project file to exclude the whole project from being instrumented. Live Unit Testing will still run the tests that haven't been instrumented, but their coverage will not be visualized.

You can also check whether Microsoft.CodeAnalysis.LiveUnitTesting.Runtime is loaded in the current application domain and disable tests based on why. For example, you can do something like the following with xUnit:

[ExcludeFromCodeCoverage]
public class SkipLiveFactAttribute : FactAttribute
{
   private static bool s_lutRuntimeLoaded = AppDomain.CurrentDomain.GetAssemblies().Any(a => a.GetName().Name ==
                                            "Microsoft.CodeAnalysis.LiveUnitTesting.Runtime");
   public override string Skip => s_lutRuntimeLoaded ? "Test excluded from Live Unit Testing" : "";
}

public class Class1
{
   [SkipLiveFact]
   public void F()
   {
      Assert.True(true);
   }
}

Your solution can build even if you're not making edits if the build process generates source code that's part of the solution itself, and your build target files don't have appropriate inputs and outputs specified. Targets should be given a list of inputs and outputs so that MSBuild can perform the appropriate up-to-date checks and determine whether a new build is required.

Live Unit Testing starts a build whenever it detects that source files have changed. Because the build of your solution generates source files, Live Unit Testing gets into an infinite build loop. If, however, the inputs and outputs of the target are checked when Live Unit Testing starts the second build (after detecting the newly generated source files from the previous build), it breaks out of the build loop because the inputs and outputs checks indicate that everything is up-to-date.

You might not see icons in the editor if the assemblies that Live Unit Testing is operating on aren't instrumented for any reason. For example, Live Unit Testing isn't compatible with projects that set <UseHostCompilerIfAvailable>false</UseHostCompilerIfAvailable>. In this case, your build process needs to be updated to either remove this setting or to change it to true for Live Unit Testing to work. 

You can do several things to collect more detailed logs:

• Go to Tools > Options > Live Unit Testing and change the logging option to Verbose. Verbose logging causes more detailed logs to be shown in the Output window.

• Set the LiveUnitTesting_BuildLog user environment variable to the name of the file you want to use to capture the MSBuild log. Detailed MSBuild log messages from Live Unit Testing builds can then be retrieved from that file.

• Set the LiveUnitTesting_TestPlatformLog user environment variable to 1 to capture the Test Platform log. Detailed Test Platform log messages from Live Unit Testing runs can then be retrieved from [Solution Root]\.vs\[Solution Name]\log\[VisualStudio Process ID].

• Create a user-level environment variable named VS_UTE_DIAGNOSTICS and set it to 1 (or any value) and restart Visual Studio. Now you should see lots of logging in the Output - Tests tab in Visual Studio.

No, you should not open or edit the files under the build and test directories of the workspace folder. Live Unit Testing should manage all of the files in the src folder, to keep them in sync between the Repository Root and Workspace Root.

Yes, but you need to make sure it's enabled. If you are using a Dev Drive, make sure that the projected file system (ProjFS) filter is enabled. For example, the following command enables ProjFS and Windows Defender:

fsutil devdrv setfiltersallowed PrjFlt, WdFilter

See also

• Live Unit Testing