Running tests in Team Build - 2
How do I enable code coverage in my build?
Code coverage can be enabled by adding the desired assembly to instrument in the Code Coverage tab of the test run configuration editor. You can just add the assemblies that must be instrumented, save the test run configuration file, and check in into the team project. Now, upon kicking off a build, code coverage results will be published in the build report. Currently code coverage data is shown per test run. The report shows the number of lines covered, not covered, and partially covered.
Note that code coverage results are published asynchronously. That means, the build report shows the build progress up until completion of build; but even after the build report notifies that the build is complete, the coverage data may not have been published fully. Let us delve a little deeper into why this happens.
On the application tier of the Team Foundation Server as it is known, there is a service installed named “Code Coverage Analysis.”This service is responsible for publishing the code coverage data into the database on the data tier. The build report displays data that is pulled out of this database. When any code coverage data is to be published in the database, this service is started and the coverage data is generated. After generation, this data is published into the database. Based on the size of the assemblies instrumented, this process can take a few seconds to a few hours. Since this should not affect the build time, this process is done asynchronously with the build process. After publishing is complete, the service is shut down again.
How do I run tests from multiple test projects?
Through the build type wizard, we select test lists to run for only a single test metadata file. But a simple edit of the build type can enable you to run tests from multiple test projects. Open the build type configuration file named TFSBuild.proj. For example, the TFSBuild.proj filecontains the following data when test lists List11 and List12 are chosen to be run.
Now suppose you want to enable the build to run test lists List21, List22, and List23 from a test project TestProject2. TFSBuild.proj must be edited with the following data:
Upon checking in the TFSBuild.proj file into source control, a build is kicked off on the edited build type. The build report now shows the results of both test lists run.
How do I run different sets of tests on different flavors of builds?
Similar to the above situation, you can check out the build type configuration file, TFSBuild.proj and perform the edits shown below. Consider a case where you wants to run test list “ListDebug” on a build built with “Debug” configuration and platform “Any CPU” whereas he wants to run “ListRelease” on a build built with “Release” configuration and platform “Any CPU.”
When a build is kicked off on the edited build type, respective tests are run on the corresponding flavors
What about deployment of extra binaries?
In the case extra binaries need to be deployed for tests to run successfully, the assemblies can be deployed in one of the following ways:
- In the Deployment tab in the test run configuration editor, add the binary to be deployed.
- Add a DeploymentItem attribute to the test method and specify the assembly path as an argument.
- In the case of generic tests, deployment items can be added as part of the test in the Additional files to deploy section in the test itself.
The deployed binaries might contain the full path to themselves when specified through any of the ways above; however, this full path will not lead to the assembly when the tests are run as part of Team Foundation Build since the folder structure of the test binaries will differ on the build machine as compared to those of the client.
Binaries on the client may be similar to:
- D:\Abc
- D:\Abc\MySolution\Sln file, test metadata file and configuration file
- D:\Abc\MySolution\MyClassLibrary\Library Files file
- D:\Abc\MySolution\MyClassLibrary\Bin\Debug\Library Binaries
- D:\Abc\MySolution\MyClassLibrary\
- D:\Abc\MySolution\MyClassLibrary\Bin\Debug\Library Binaries
- D:\Abc\MySolution\MyWebsite\Web pages
- D:\Abc\MySolution\MyWebsite\App Code\Web Code Files
- D:\PreCompiledWeb\Bin\Web Binaries
- D:\Abc\MySolution\MyTestProject\Test project files
- D:\Abc\MySolution\MyTestProject\Bin\Debug\Test Binaries
The build computer contains files similar to the structure below (where D:\Build is the build directory):
- D:\Build\TeamProject\BuildType\Sources\MySolution
- D:\Build\TeamProject\BuildType\Sources\MySolution\Sln file, test metadata file and configuration file
- D:\Build\TeamProject\BuildType\Sources\MySolution\MyClassLibrary\Library Files
- D:\Build\TeamProject\BuildType\Sources\MySolution\MyWebsite\Web pages
- D:\Build\TeamProject\BuildType\Sources\MySolution\MyWebsite\App Code\Web Code Files
- D:\Build\TeamProject\BuildType\Sources\MySolution\MyTestProject\Test Project Files
- D:\Build\TeamProject\BuildType\Binaries\Debug\_PublishedWebSites\MyWebsite\Web Binaries
- D:\Build\TeamProject\BuildType\Binaries\Debug\Test Binaries
- D:\Build\TeamProject\BuildType\Binaries\Debug\Library Binaries
Therefore, the test binaries as well as deployed files are searched in a different set of locations when tests are run through Team Foundation Build. These locations are also output in the log file generated. Typically, files are searched for in the Binaries folder in the build directory recursively. Test types other than unit tests require the test files themselves unlike the unit tests that require only test binaries to run. So, one of the folders searched would be the Sources folder itself. If the binary is not found in either location, the full path specified is searched.