Share via


How to: Create Test Cases from an Assembly of Automated Tests Using tcm.exe

If you have created automated tests using Visual Studio, you might want to run these tests as part of a test plan. By adding them to a test plan, you can measure your testing progress. You can view the results from these automated tests and any other tests in your test plan together and track the progress that you are making. You can also link these test cases to requirements, so that you can view the status of automated tests for the requirement. For more information about how to do this, see Link Existing Test Cases to a Requirement.

To run these tests as part of a test plan, you can use a command line tool tcm.exe to create test cases for all your test methods in an assembly or some of your test methods based on test method properties or the test category. If you want to add a test to an existing test case, you can associate a test method directly with that test case. For more information, see How to: Associate an Automated Test with a Test Case.

Requirements

  • Visual Studio Ultimate, Visual Studio Premium, Visual Studio Test Professional

If you have existing automated tests from earlier versions of Visual Studio, you can associate these tests with a test case after you upgrade the test project that contains the automated tests to Visual Studio.

After you have imported your test methods and created test cases, you have to add these test cases into your test plan. To run these test cases from your test plans, you must use a lab environment. You cannot run automated tests using Microsoft Test Manager without an environment.

You must check in the test project that contains the automated test methods and make sure that the test project is part of your build definition. You must make sure that a build created by using this build definition is selected in your test plan. The information from the build is used by Microsoft Test Manager to locate the correct test method assembly and use this to run the automated tests.

For more information about how to check in your test project and team build, see How to: Add a Project or Solution to Version Control and Building the Application.

You can also run automated tests from the command line using test settings and a test environment. For more information, see How to: Run Automated Tests from the Command Line Using Tcm.

Use the following procedures to create test cases from an assembly of automated tests that you can then run as part of your test plan:

  • Create the test cases from your assembly of automated tests and add them to your plan

  • Set up your test plan to use your team build

  • Create your test settings and environment to run your tests

  • Run the automated test using Microsoft Test Manager

  • View and update the test results

Common Parameters for the Procedures

Parameter

Description

/collection:CollectionURL

Specifies the URI of the team project collection. The format for the URI is as follows: http://ServerName:Port/VirtualDirectoryName/CollectionName

If no virtual directory is used, then the format for the URI is the following:

http://ServerName:Port/CollectionName

If you do not know the correct path, contact your Team Foundation system administrator.

/teamproject:TeamProjectName

The team project that you want to use. This team project must be defined in the project collection that is specified by the /collection parameter.

/login:UserName,Password

Optional. Specifies the name and password of a user who is logged on to the application-tier server for Team Foundation Server and who has permissions to run the command.

You would use this option if your Windows credentials do not have the appropriate permissions, or you are using basic authentication, or you are not connected to a domain.

/? or help

Displays help about the command in the Command Prompt window.

Procedures

Create Test Cases from an Assembly of Automated Tests

To create test cases from an assembly of automated tests

  1. Open a Visual Studio command prompt.

    To do this, choose Start, point to All Programs, point to Microsoft Visual Studio 2012, point to Visual Studio Tools, and then choose Developer Command Prompt.

    The command prompt opens to the folder: <drive>:\Program Files\Microsoft Visual Studio 11.0\VC

  2. Change directory to the folder that contains the assembly built from your test project.

    To do this, first change directory to your solution folder. Then change directory to the folder for your test project by typing the following command at the command prompt:

    cd <project name>\bin\Debug

  3. To import all the test methods from your assembly, type the following at the command prompt:

    tcm testcase /collection:<CollectionURL> /teamproject:<Team Project name> /import /storage:<name of your assembly>.dll

    You will see results similar to these:

    Modification Title

    ---------------- ------------------

    Added <Name of test method>

    Added <Name of test method>

    Summary

    ------------

    Added 2

    Updated 0

    Total 2

    Note

    The test cases that are created will have the same names as the test methods that were imported.

  4. (Optional) You can limit the test methods that are imported from the assembly based on test categories. For more information about test categories, see Defining Test Categories to Group Your Tests. For example, the following command shows you how to import tests that are in both the ShoppingCart and SmokeTest categories.

    tcm testcase /collection:<CollectionURL> /teamproject:<Team Project name> /import /storage:<name of your assembly>.dll /category:"ShoppingCart&SmokeTest"

  5. (Optional) You can limit the test methods that are imported from the assembly based on a test method property. For example, the following command shows how to import all test methods that have priority set to 0 or 1.

    tcm testcase /collection:<CollectionURL> /teamproject:<Team Project name> /import /storage:<name of your assembly>.dll/maxpriority:1

  6. Open Microsoft Test Manager.

    Note

    To display the Microsoft Test Manager window, choose Start, and then choose All Programs. Point to Microsoft Visual Studio 2012 and then choose Microsoft Test Manager.

  7. To select a test plan, choose the down-arrow on the center group switcher and then choose Testing Center.

  8. On the center group menu bar, choose Plan.

  9. (Optional) To select a different test plan, choose the test plan name hyperlink in the upper-right corner, or choose the home icon.

  10. You can add the test cases to a new test suite, or an existing test suite. To add the test cases into your plan, choose Contents and then select the test suite in the test suite hierarchy or add a new test suite. Then choose Add in the test suite details pane.

    The Add Test Cases to Suite dialog box is displayed.

    Note   For more information about test suites in test plans, see Organizing Test Cases Using Test Suites.

  11. Select the test cases that were created. These test cases will have the same names as the test methods in the test assembly. Then choose Add Test Cases.

    The test cases are shown in the list of tests for the test suite that you selected.

    Note

    You can also import your test cases directly into a static test suite. For more information about how to do this, see tcm: Importing Automated Tests into Test Cases.

Set Up Your Test Plan To Use Your Team Build

To set up your test plan to run the automated test that you have created, you must choose the correct build definition used to build your automated test methods. You must do this so that the automated test assembly can be found in the share location for your build definition and then it can be run from Microsoft Test Manager.

To set up your test plan to use your team build

  1. Open Microsoft Test Manager.

    Note

    To display the Microsoft Test Manager window, choose Start, and then choose All Programs. Point to Microsoft Visual Studio 2012 and then choose Microsoft Test Manager.

  2. To select a test plan, choose the down-arrow on the center group switcher and then choose Testing Center.

  3. On the center group menu bar, choose Plan.

  4. To set up your test plan to run the automated tests, choose Properties and then choose Filter for builds.

    The Filter completed builds dialog box is displayed.

  5. To select the build definition that is used to build your automated tests, choose Build definition.

  6. Each build can be given a specific value to reflect the quality of the build. To select the quality of the builds you want to be able to view, choose Build quality.

    Note

    For more information about build definitions and build quality, see Define Your Build Process.

  7. To save your changes, choose Set build filter.

  8. To select the most recent build to use with this test plan that includes the latest changes to the automated test, you must first choose Save to save the plan and then choose Modify.

    The Assign Build activity is displayed. You can compare your current build with a build you plan to take. The associated items list shows the changes to work items between the builds. You can then assign the latest build to use for testing with this plan. For more information about how to select a build, see Determining Which Builds Have Bug Fixes, New Features, or Requirements.

  9. To close the Assign Build activity and return to the test plan properties, choose the Close icon.

  10. To save these changes for this test plan, choose Save in the toolbar.

Create Your Test Settings and Environment to Run Your Tests

To run your automated tests, you must use a lab environment. You cannot run automated tests using Microsoft Test Manager without an environment.

You must create an environment that contains the roles in your test settings and then use this environment in your test plan. For more information about how to create your environment and roles and test settings, see Setting Up Test Machines to Run Tests or Collect Data.

Note

If you want to run automated tests that interact with the desktop, you must set up your agent to run as a process instead of a service. For more information, see How to: Set Up Your Test Agent to Run Tests that Interact with the Desktop.

To create your test settings and environment to run your tests

  1. To create a standard environment, follow the steps in this topic: Creating a Standard Environment.

  2. To create an SCVMM environment, follow the steps in this topic: Creating an SCVMM Environment. If you want more information about how to create lab environments, see Managing Stored SCVMM Environments and Virtual Machines.

  3. To create your test settings, follow the steps in this topic: Create Test Settings for Automated Tests Using Microsoft Test Manager.

Run the Automated Test Using Microsoft Test Manager

To run the automated test using Microsoft Test Manager

  1. Open Microsoft Test Manager.

    Note

    To display the Microsoft Test Manager window, choose Start, and then choose All Programs. Point to Microsoft Visual Studio 2012 and then choose Microsoft Test Manager.

  2. To run the automated test, choose the down-arrow on the center group switcher and then choose Testing Center.

  3. In the center group menu bar, choose Test.

  4. (Optional) To override the build, the test settings or the environment to use for running the automated tests that you select in this test plan, right-click the test and then choose Run with options. For example, if you want to run on a staging environment instead of your standard testing environment then you might choose a different environment. From the Run options dialog box, you can change these settings, and then choose Run to run the selected test.

    Note

    If you select a different environment, it must have the same roles as the environment that was created in the test settings that you use.

  5. To run the automated test without changing any options, right-click the test and then choose Run.

    The Analyze Test Runs activity is displayed that shows the progress of the test run that contains this test.

    Note

    You can run multiple automated tests by selecting multiple tests, or you can select to run a whole suite of tests. To run a suite, right-click the test suite and then choose Run.

View and Update the Test Results

To view and update the test results

  1. Open Microsoft Test Manager.

    Note

    To display the Microsoft Test Manager window, choose Start, and then choose All Programs. Point to Microsoft Visual Studio 2012 and then choose Microsoft Test Manager.

  2. To view the test results, choose the down-arrow on the center group switcher and then choose Testing Center.

  3. On the center group menu bar, choose Test and then choose Analyze Test Runs.

    The Analyze Test Runs activity is displayed. It shows any test runs for this test plan.

  4. Double-click a test run to open it and view the details.

    The test run details are displayed.

  5. (Optional) To update the title of your test run to be more meaningful, type the new name in Title.

  6. (Optional) If your test failed, you can update the reason for the failure. Choose Resolution and select the reason for the failure from the list.

  7. (Optional) To add comments to the test result, choose the Comments icon. Type your comments and then choose Save comments.

  8. (Optional) To view the details of an individual test, double-click the test.

    The test result is displayed. It shows the details from the test run, the attachments for data collected for this test result, and the test results history for that test. You can close this view to return to the test run.

    Note

    If you determine that there is a bug from your analysis, you can create a bug from this view.

  9. To save these changes for this test run, choose Save in the toolbar.

See Also

Tasks

How to: Associate an Automated Test with a Test Case

Reference

tcm: Importing Automated Tests into Test Cases

Concepts

How to: Run Automated Tests from the Command Line Using Tcm

Other Resources

Creating Automated Tests