How to: Run Automated Tests from a Build-Deploy-Test Workflow
Note
This topic applies when you are testing in both standard environments and SCVMM environment. However, the steps that involve environment snapshots only apply to SCVMM environments.
You can run automated tests in a lab environment using a build-deploy-test workflow by queuing the build for the workflow, or by configuring the workflow to run at a scheduled interval. For information on creating a build-deploy-test workflow and scheduling your workflow, see Setting Up Automated Build-Deploy-Test Workflows. For example, you can build and deploy daily builds of your application on a server and run tests on that server using a client. After you run your automated tests, you can analyze the test results to check the quality of your builds. Test results are saved every time your build-deploy-test workflow is completed. You can use these results to view how stable your build is and decide whether to use that build for manual testing.
If you’re using an SCVMM environment, you can also use your build results to connect to a snapshot of your lab environment that was made after your application was deployed. This enables you to view the state of the environment after a test failure, even after a new test run restores your environment with a clean snapshot.
This topic describes how to run or schedule a build-deploy-test workflow to run your automated tests in a standard environment or an SCVMM environment, and then view your test results. In addition, this topic describes how to connect a post deployment snapshot from your build results when using an SCVMM environment.
Use the following procedures to run automated tests on your application in a lab environment using a build-deploy-test workflow, and then analyze the test results:
Check the Prerequisites
Start Your Build
Connect to the Environment from Your Build Results
View and Analyze the Test Results For Your Build
Prerequisites
Before you configure and run your scheduled tests in a lab environment, use this list to verify that you have completed the following tasks:
Prerequisite Tasks
Configure a test controller. See Setting Up Test Controllers in Lab Environments with Microsoft Test Manager.
Configure a build controller and build agent using Team Foundation Server Administration. See How to Configure a Build Machine.
Create a lab environment and configure it with the required roles for your application. See Creating Lab Environments.
(Recommended) If you’re using an SCVMM environment, take a snapshot of your environment to use as a clean state for your workflow. See Using Environment Snapshots with SCVMM Environments.
Add your automated tests to a test plan. See How to: Associate an Automated Test with a Test Case.
Create test settings for your automated tests and assign the tests to the roles in your lab environment. See Create Test Settings for Automated Tests Using Microsoft Test Manager.
Create your build-deploy-test workflow. See Setting Up Automated Build-Deploy-Test Workflows.
Start Your Build
You can run your automated tests from a workflow by queuing a build.
To queue your build
To start build-deploy-test workflow, open the shortcut menu for your build definition in the Builds folder, and choose Queue New Build.
The Queue Build dialog box is displayed.
Verify the information for your workflow and then choose Queue.
The Build Explorer view is displayed.
To see the Build Summary view as the build progresses, double-click your build.
You can see the status as the build progresses.
(Optional) If you want to view the lab environment as the build progresses, open Microsoft Test Manager, locate the Lab Center, choose Lab, and then choose your environment in the list.
If your build-deploy-test workflow is completed successfully, you’ll see a green check mark. If there are errors, you can choose View Log to see details.
Connect to the Environment from Your Build Results
You might want to connect to your lab environment to investigate an issue if a test fails during the workflow process. You can connect either to the post deployment snapshot, if you selected this option in your build workflow and are using an SCVMM environment, or to the environment in its current state, as shown in this illustration:
.png "Connect to the Environment from Your Build Results")
To connect to the environment from your build results
From the Builds folder in Team Explorer, open the shortcut menu for your build definition that contains your workflow and point to View Builds.
The Build Explorer view is displayed.
To view your completed build, choose the Completed tab.
Double-click the build that you want to view.
The Build Summary view is displayed.
Choose the link next to View environment snapshot <Build name and number>.
The Connect to environment dialog box is displayed.
-
Note
This step only applies to SCVMM environments. If you’re using a standard environment, skip this step.
If you’re using an SCVMM environment, and you want to connect to the snapshot that was taken after the application was deployed, choose Connect to the snapshot in this environment.
Note
By connecting to this snapshot, any changes that were made after this post-deployment snapshot will be discarded. If you want to keep any changes, connect to the environment in its current state and take a snapshot first, before reverting to the post-deployment snapshot. For information about how to take a snapshot, see How to: Save the Current State of Your Environment.
If you want to connect to the environment in its current state after you have run any tests from your workflow, choose Connect to the environment in its current state.
Choose Connect.
The Microsoft Environment Viewer is displayed and you are connected to the environment. You can now investigate any issues.
View and Analyze the Test Results for Your Build-Deploy-Test Workflow
You can view the test results summary in your build summary. However, you can also view and analyze the test results by using Microsoft Test Manager because the results are stored as part of your test plan. This is shown in the next illustration. See Reporting on Testing Progress for Test Plans.
.png "View the Test Results from Your Build Workflow")
To view and analyze the test results from Microsoft Test Manager
Open Microsoft Test Manager.
Note
To display the Microsoft Test Manager window, choose Start, and then choose All Programs. Point to Microsoft Visual Studio 2012 and then choose Microsoft Test Manager.
To view the test results, choose the down-arrow on the center group switcher and then choose Testing Center.
On the center group menu bar, choose Test and then select one of the test suites from the test suite hierarchy that you used in your build workflow.
You can see the results for the tests for the configuration that you selected in your build workflow.
If you want to analyze the complete test run, choose Analyze Test Runs.
The Analyze Test Runs activity is displayed. It shows any test runs for this test plan.
Note
The run title will reflect the name of your build definition. The run ID is displayed in the build summary page for your build to help you identify the run.
Double-click a test run to open it and view the details. The test run details are displayed.
(Optional) To update the title of your test run to be more meaningful, type the new name in Title.
(Optional) If your test failed, you can update the reason for the failure. Choose Resolution and select the reason for the failure from the list.
(Optional) To add comments to the test result, choose the Comments icon. Type your comments and then choose Save comments.
(Optional) To view the details of the individual test, double-click the test.
The test result is displayed. It shows the details from the test run, the attachments for data that was collected for this test result, and the test results history for that test. You can close this view to return to the test run.
Note
If you determine that there is a bug, you can create a bug from this view.
To save any changes for the test run, choose Save in the toolbar.
For more information on viewing test results, see How to: View Test Plan Results in Microsoft Test Manager.