How to: Configure and Run Build Verification Tests (BVTs)

A build verification test (BVT) typically consists of a broad suite of tests that are used to verify the overall quality of a particular build. BVTs are usually automated and run on a set schedule, frequently at night. They can also be run manually, such as after an automated run has failed. A build is considered a success if all the tests in the BVT have passed.

BVTs use different areas of Visual Studio Team System: Visual Studio Team System Test Edition, Team Foundation version control, and Team Foundation Build. First you use Test Edition to create a test list that contains the tests you want included in your BVT. Then you check in that test list to source control, using Team Foundation version control. Finally you use Team Foundation Build to run the BVT, or to schedule regular runs of the BVT.

Note

Team Foundation Build and Team Foundation version control are not a part of Test Edition. To use the features of these products, and in particular to perform the steps in most of the following procedures, you must have installed either all of Team System or the individual products Team Foundation version control and Team Foundation Build. Additionally, the client computer must have Team Explorer installed, and your Visual Studio user session must be connected to a Team Foundation Server computer. For information about how to connect to Team Foundation Server, see How to: Connect to Team Foundation Server.

This topic describes all the procedures that are required to create, store, and run build verification tests:

• Create a BVT Test List. Create a test list and populate it with the tests your BVT requires. For more information, see Create a BVT Test List.

• Check in the BVT. Add your solution and the BVT to source code control. For more information, see Check in the BVT.

• Create a Build Type, specifying to run the BVT test list as part of the build. You must first create a build type before you can run builds with Team Foundation Build. For more information, see Create a Build Type.

• Run the BVT Build Type. Run the build; this automatically also runs the BVTs. For more information, see Run the BVT Build Type.

Create a BVT Test List

To create a BVT test list in the Test List Editor

1. In the Test List Editor, click By Test List to display the test list pane.

2. Right-click Lists of Tests and then click New Test List.

   This displays the Create New Test List dialog box.

3. Type the name of the test list that will contain your build verification tests; for example, type BVTs.

4. (Optional) Type a description of your BVT test list, and select a location in the test list hierarchy to put the new test list.

5. Click OK.

   Your new BVTs test list is created and appears in the test list hierarchy pane.

6. Click All Loaded Tests to display all the tests in your solution.

7. Select the tests that you want to run as your build verification tests and drag them into the new BVTs test list.

   Note

   To select tests for dragging, click their rows in the Test List Editor. To select multiple tests, use SHIFT+click and CTRL+click. Do not use the check boxes for this purpose.

8. (Optional) To verify the contents of the test list, click it in the test list hierarchy pane.

   This BVTs test list, together with all the test lists in your solution, resides in your solution's test metadata file. Test metadata files have the extension .vsmdi. For more information, see Using Test Lists.

Check in the BVT

In this procedure, you check in all the files of your solution. This includes the test metadata file of your solution, which contains your test lists and tests. Whenever you add, delete, or reorganize test lists, or change the contents of test lists, your test metadata file is automatically updated to reflect those changes.

To check in the BVT to source control

1. Connect to a Team Foundation Server computer. For more information, see How to: Connect to Team Foundation Server.

2. If your solution is not already in source control, add it to source control. For more information, see How to: Add a Project or Solution to Version Control.

3. Click View and then click Pending Checkins to display the Pending Checkin window.

4. Check in the all the files of your solution. For more information, see How to: Check In Pending Changes.

   Note

   If you are using Visual Studio Team System, you might have a specific team process that governs how BVTs are created and managed. For example, the process might require that you verify your build locally before you check in that code and the tests that run on it.

   After the check-in operation is finished, a padlock icon is displayed next to each file in Solution Explorer to indicate its Checked In status. For more information, see How to: Identify Source Control Item Status.

   Your checked-in tests are available to the Team Foundation Build feature of Team System. You can now create a build type that contains the tests you want to run in your BVT.

Create a Build Type

To create the BVT build type

1. In the Team Explorer window, click your team project.

2. Right-click Team Builds and then click New Team Build Type.

   The New Team Build Type Creation Wizard appears.

3. Follow the steps in the New Team Build Type Creation Wizard. For more information, see How to: Create a Build Definition,

   Important Note:
   On the Select build options page in the New Team Build Type Creation Wizard, click Run Test. Then select the test metadata file that contains the BVT test list you created in Create a BVT Test List. Finally, select the test list that contains the tests you want to run, namely the tests of your BVT.

4. When you have finished entering information in the wizard, click Finish.

   Your new build type appears in the Team Explorer window under the Team Builds folder.

Run the BVT Build Type

To run the BVT using Team Build

1. On the Build menu, click Build {Team Project Name}.

2. The Build {Team Project Name} dialog box is displayed with a list of all existing build types. Make sure your build type is selected.

3. (Optional) Change the build location and directory.

4. Click Build.

   For more information, see How to: Queue or Start a Build Definition. To schedule a regular build, see How to: Configure A Scheduled Build (Command Line).

See Also

Tasks

How to: Organize Tests into Test Lists

How to: Create a Build Definition

Other Resources

Working with Team Foundation Version Control