How to: Run Manual Tests and Create Reproducible Bugs in SCVMM Environments
You can use SCVMM environments when you run your manual tests, if you have Visual Studio Lab Management. You create the virtual machines or templates for the roles that are required for your application and select this set of roles from your test settings. For more information about environments, test settings and roles, see Setting Up Test Machines to Run Tests or Collect Data and Setting Up Machines and Collecting Diagnostic Information Using Test Settings.
When you set up your SCVMM environment to run manual tests, you create virtual machines or templates for the backend roles of your application. For example, you could create a SCVMM environment that includes virtual machines or templates for the Web server and database server roles in your environment. Then you could run the manual tests from a desktop computer that is not part of the environment. Or, you might run a browser to connect to your Web server from this computer, as shown in the following illustration.
Requirements
- Visual Studio Ultimate, Visual Studio Premium, Visual Studio Test Professional
.png "Run Manual Tests from a Virtual Environment")
Note
If you want to use this same environment to run automated tests, you might want to add a role for the desktop client and use a virtual machine or template for that role. You cannot use this role to run your manual tests. However, you can use this role when you run automated tests. For more information about how to run automated tests with lab environments, see How to: Run Automated Tests In a Lab Environment Using Microsoft Test Manager.
If you find a bug in your application when you run your manual tests, you can take a snapshot of your SCVMM environment that developers can use to connect to the actual state of your servers in your environment when you found the bug. This helps reduce the number of bugs that cannot be reproduced by developers. This state of the environment is saved for developers to use to investigate whenever it is convenient, and you can continue to run more tests on this same environment. When developers want to use the environment, they can connect to the snapshot by using the link in the bug.
Note
This snapshot is a snapshot of each virtual machine in the environment. These snapshots are stored on the physical host that is used for the virtual machine. The snapshots do not use physical space on Team Foundation Server. In the bug or the test results, only the link to the snapshot is added.
If the environment might be deleted before the developer checks the bug, the tester can store a copy of the environment in the library share when they have completed testing with it. The link in the bug will allow the developer to create a copy of the stored environment and use this.
Note
If you want to create a copy of the environment using the link in the bug, the environment must be a network isolated environment.
If you want to build and deploy the backend tiers of your application to your SCVMM environment, you can install them manually or use a build-deploy-test workflow to deploy them as part of a build definition. To use a build-deploy-test workflow, you need a build controller. See How to: Create a Build-Deploy-Test Workflow for an SCVMM Environment.
Prerequisites
Before you can run manual tests using your environment you must follow these steps:
Install Visual Studio Ultimate, Visual Studio Premium, or Visual Studio Test Professional on any of the machines that you want to use to run your manual tests.
Create an SCVMM environment that includes a machine for each role that is required in your tests. See Creating an SCVMM Environment.
Create your manual test cases by using Microsoft Test Manager and add them to a test suite in a test plan. For more information, see Defining Your Testing Effort Using Test Plans.
Procedures
Use the following procedures to enable you to run your manual tests and create reproducible bugs. If you find an issue, you can connect to the virtual machines in your environment to verify information, or take a snapshot of the environment and save it in a bug so that a developer can use the environment to help reproduce the bug.
Run manual tests using SCVMM environments
Connect to the virtual machines in your environment
Create a snapshot of your virtual environment
Connect to a snapshot of an SCVMM environment
Run Manual Tests Using SCVMM Environments
To run manual tests using SCVMM environments
Create an SCVMM environment and add the necessary virtual machines or templates for each role. For more information, see Creating Lab Environments.
To start the SCVMM environment, select the environment from the Environments view and choose Start.
Verify that the machines in your environment have started and that the status of the environment is Ready.
Note
If the environment that you want to use has been stored in the library, you must deploy the environment to create the virtual machines from the stored images before you can start the environment.
If the status of the environment is Not Ready, choose Repair on the shortcut menu. This step makes sure that a test agent is installed on each machine, so that it can be managed by the test controller.
(Recommended) To prevent other users from accidentally connecting to your environment when you run your tests, choose the drop-down arrow next to Mark 'In Use' in the toolbar. Add a comment about who is using the environment and choose Mark 'In Use'.
If another user tries to connect to the environment that is being used, a warning with the comments that you entered is displayed in the Connect to environment dialog box. However, this user can still connect to the environment and ignore the warning.
Note
You can also use the Microsoft Environment Viewer to mark which environment is being used.
Install the backend tiers of your application on the virtual machines in your environment. To connect to the backend tiers, you might also have to install the local client for your application on your desktop computer. You can use the workflow integration capability to install the appropriate backend tiers of your application automatically by using a build definition. For more information about how to do this, see How to: Create a Build-Deploy-Test Workflow for an SCVMM Environment. Or, you can install the application manually.
Note
We recommend that you take a snapshot of your environment after you install your application so that you can easily return to this clean state with the application installed.
Create test settings that contain the set of roles for your lab environment by using Microsoft Test Manager and select the data and diagnostics that you want to collect for each role. For more information about how to create test settings for manual tests, see Create Test Settings for Manual Tests Using Microsoft Test Manger.
To assign the test settings to your lab environment, follow these steps:
Using Microsoft Test Manager from the machine where you want to run your tests, choose the down-arrow on the center group switcher and then choose Testing Center.
On the center group menu bar, choose Plan and then choose Properties.
The properties for the currently selected test plan are displayed.
Choose the drop-down arrow for Test settings under Manual runs and select the test setting.
Choose the drop-down arrow for TestEnvironment under Manual runs and select the environment.
Choose Save and Close.
To run your tests, choose Test, choose Run Tests. Select the tests that you want to run, and choose Run.
To run your tests using the test settings that you created for this specific test run only, follow these steps:
Using Microsoft Test Manager from the machine where you want to run your tests, choose the down-arrow on the center group switcher and then choose Testing Center.
To run your tests, choose Test, choose Run Tests. Right-click the test suite or tests that you want to run, and choose Run with options.
The Run Options dialog box is displayed.
Select the test settings that you created from Test Settings under Manual test runs.
Select the lab environment from Environment under Manual test runs and choose Run.
Test Runner is now displayed with the first test selected.
To start the first test, choose Start Test. For more information about how to run manual tests, see How to: Run Manual Tests.
You can now begin to run all the tests that you selected.
To run the steps in your test, you must start your application under test. To connect to the part of the application that is installed on your environment, you use the client for your application. This could be a desktop client or a Web browser that is installed on the computer where you have installed Microsoft Test Manager.
If you are using an environment that is network-isolated, see the following topic for more information about connecting to your application, see the section about Communicating Between Machines in Network-Isolated Environments and Computers in the External Network in the following topic: How to: Create and Use a Network Isolated Environment.
To save your test results, choose Save and Close from Test Runner.
The Run Tests view is displayed.
Connect to the Virtual Machines in the Environment
If you want to verify some information about a machine in your lab environment when you run your tests, you can connect to any of the virtual machines. For example, you might want to verify that the value of a registry entry is correct.
To connect to the virtual machines in the environment
To view the state of a machine in your lab environment when you run a test, you can choose Connect to environment in the test steps toolbar for Test Runner.
The Microsoft Environment Viewer is displayed. You can select the virtual machine that you want to view. You can also run commands on the virtual machine.
For more information about how to connect to machines in your environment, see How to: Connect to a Lab Environment.
If you are using an environment with network isolation, see How to: Create and Use a Network Isolated Environment.
Note
You must not restore to a snapshot when you connect to an environment while you are running tests. If you restore to a snapshot while tests are running, the test agents will be restarted and the data and diagnostics cannot be collected for the test run. If you want to restore a snapshot, you must stop the test run and restore to a snapshot before you start to run tests.
Create a Snapshot of Your SCVMM Environment
To create a snapshot of your SCVMM environment
If you find an issue when you run your test, choose the Take Environment Snapshot icon at the far right of the toolbar for the test steps panel. Now when you create a bug, the link to this snapshot of the lab environment is added to the bug.
Note
This takes a snapshot of each virtual machine in the environment. These snapshots are stored on the physical host that is used for the virtual machine. The snapshots do not use physical space on Team Foundation Server. In the bug or the test results, only the link to the snapshot is added.
In the Details tab for the bug, you can see the snapshot with the following name: TC<test case id>snapshot<snapshot number>.lvr. You can also see any data or diagnostics that was collected on the virtual machines for your environment in the details for the bug.
Note
You can also view this link to the snapshot in the test results.
To save your test results, choose Save and Close from Test Runner.
The Run Tests view is displayed.
Connect to a Snapshot of a SCVMM Environment
To connect to a snapshot of an SCVMM environment
To connect to the SCVMM environment snapshot from your test results, from the Run Tests view in Microsoft Test Manager, right-click a test and point to View Results.
Choose the link to the snapshot of the environment displayed in the test step when you created the snapshot.
Choose Open to open the file.
The Connect to environment dialog box is displayed.
From this dialog box, select one of the following options:
To restore the exact state of the SCVMM environment at which the snapshot was taken, choose Connect to the saved snapshot in this environment.
Note
If you select this option, you might disconnect any user who is currently using this environment and this could lead to loss of their work. You can verify that there is no warning message that the environment is in use.
To connect to the environment in its current state and not revert to the snapshot that was taken when the test was run, select Connect to the environment in its current state and choose Connect.
Note
You might still disconnect another user who is connected to this environment, but you will not change the state of the environment.
If there is a stored copy of the environment in the library share, you can create a copy of the environment and connect to that. You might want to do that if the environment is currently in use, or you do not want to prevent the testers from using the environment. To create a copy of the environment and connect to that copy, select Connect to a new instance of this environment.
Note
When you create a copy of the environment, this can take a long time and also uses significant disk space on the project library share. If this copy of the environment might be started at the same time as the tester's copy of the environment, you have to create a network isolated environment. For more information about environments, see Creating Lab Environments.
To connect to the SCVMM environment snapshot from your bug, you can open the bug from the Verify Bugs view using Microsoft Test Manager, or you can open the bug from the Team Explorer view in Visual Studio 2012. Then you can choose the link to the snapshot of the environment in the Details tab for the bug.
The Connect to Environment dialog box is displayed. Select the appropriate option as in the previous step in this procedure. If the environment has been marked in use, a warning message appears in the Connect to environment dialog box
Note
Developers can connect to the SCVMM environment from the bug to re-create the issue in the bug and help them resolve the issue more quickly. When you restore an environment to a snapshot, the network connections between the machines in the snapshot might stop working and any transactions that were in process at the time that the snapshot was taken might fail. If reproducing a bug depends on TCP connections that are open and active when the snapshot was taken, then you might not be able to reproduce the bug.