shankars

Using Excel Reports with Microsoft Test Manager to see test pass status of test cases linked to a work item

Shankar MSFT — Sun, 19 Jun 2011 19:55:37 GMT

From Microsoft Test Manager, you can create a Requirement based suite to see the associated test cases for a given user story or a requirement.

On the right side pane, under the Test Activity, you can see the test pass/ fail status for all test cases associated with this user story.

Problem Statement:

The right side pane of the Requirement based suite will only show the test case status based on whatever the tester status the tester has provided in this particular suite. If the test cases present in the Requirement based suites are also present elsewhere (in another test suite or another test plan) and you just want to see the status of these test cases (most recent pass/ fail information), Excel reports are the way to go.

Note: Using the report given below, you can see the status of test cases that are associated (linked) to a generic work item, not necessarily a user story. If some test cases are linked to each other (slightly modified versions of the same test case with different iteration paths/ area paths etc) and you want to see the status of these linked test cases, the same report can also be used.

The Excel Report:

You can create an Excel report as follows:

Report Filter:

1. Work Item Linked.[System_Title] – Select the the user story as the linked item. Use this filter to get all test cases linked to a user story

Row Labels:

1. Test Configuration.[Configuration Name] – Test Configurations selected under the report filters.

2. Test Case.[System_Title] – The names of all test cases, grouped by test configuration

Column Labels:

1. Test Result.[Outcome] – the state of the test case (failed, passed, never run, error, blocked or aborted)

Values:

1. Test. [Point Count Tread] – indicates the count of all test points (test cases x configurations) satisfying the row/ column labels

Rebuilding the TFS Warehouse for Excel reports

Shankar MSFT — Sun, 19 Jun 2011 19:26:49 GMT

You can use Excel reports with Microsoft Test Manager as described here

You need to have SQL server Analysis services turned on to make Excel reports work. The Excel gets data from the Analysis Services cube, which in turn gets data from the TFS Warehouse on a periodic basis.

Since test results, test plan and test suites are not work items, queries in Visual Studio cannot be used to build equivalent reports.

If you already have SQM Server Analysis services (installed along with TFS), to rebuild the data warehouse and the Analysis Services database

1. Open the administration console for Team Foundation.

2. In the navigation bar, click Reporting.

3. In Reporting, click Start Rebuild.

4. In the Rebuild the Warehouse and Analysis Services Databases dialog box, click OK.

Note: The warehouses will continue to be rebuilt and the data will continue to be repopulated after the Start Rebuild action finish. Depending on the size of your deployment and the amount of data, the whole process might take several hours to complete.

You can also use the RebuildWarehouse command (msdn.microsoft.com/.../ee349264.aspx)

To configure Warehouse settings

Browse to https://servername:8080/tfs/TeamFoundation/Administration/v3.0/WarehouseControlService.asmx and use the GetSetting and ChangeSetting webservices to get/ set warehouse rebuild settings.

Using Excel reports with Microsoft Test Manager to identify test state at a test plan level

Shankar MSFT — Fri, 04 Mar 2011 06:23:00 GMT

From Microsoft Test Manager, it is possible to identify the state of test cases (pass, fail, blocked etc) for a particular test suite. You can just select the test suite on the left side and the right pane would show all test cases with their test status.

Figure 1 – Selecting a test suite shows the state of all test cases in that test suite

However, getting the same information at a test plan level is not easy. In the Test Plan properties view, you can see a pie chart indicating the state of all test cases in a given test plan.

Figure 2 – Pie chart in test plan properties indicating overall state of all test cases in the test plan

But, it is not possible to drill down deeper to identify which if the test cases are failing in the test plan.

The workaround – Creating your own Excel Reports

The workaround is to create a report in Excel. In Excel, you can create a new report as shown below:

Figure 3 – Excel report showing state of all test cases (grouped by configurations) in a test plan

Figure 4 – You can drill down to identify failing test cases in a particular test plan

Steps to create this Excel Report

Figure 5 – The PivotTable used to create the Excel Report

Once you’ve created a PivotTable report (steps 1 to 6 given here), you can create a report with the following fields:

Report Filter:

1. Team Project. [Team Project Hierarchy] – Select the team project for the report

2. Test Suite.[Test Suite Hierarchy] – Drill down to select the test plan for the report.

Row Labels:

1. Test Configuration.[Configuration Name] – Test Configurations selected under the report filters.

2. Test Case.[System_Title] – The names of all test cases, grouped by test configuration

Column Labels:

1. Test Result.[Outcome] – the state of the test case (failed, passed, never run, error, blocked or aborted)

Values:

1. Test. [Point Count Tread] – indicates the count of all test points (test cases x configurations) satisfying the row/ column labels