Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Azure DevOps Services
Azure Test Plans provides the functionality to customize the failure types of any test case beyond the default values. A failure type is an artifact that helps to mark test case failures into defined categories such as regression issue or known issue. While this categorization of failure types is helpful, users might want to add their own custom failure type beyond the default values, which creates a more customized experience for specific user needs in the Azure DevOps project. This article walks through the steps of defining a custom failure type using the new REST API provided.
Prerequisites
| Category | Requirement |
|---|---|
| Project access | Project member. |
| Access levels | - To view test-related work items or run test cases: At least Basic access. Users with Stakeholder access have no access to Test Plans. - To exercise the full range of test-related features: Basic + Test Plans access level or one of the following subscriptions: - Enterprise - Test Professional - MSDN Platforms |
| Permissions | - To add or modify test plans, test suites, test cases, or other test-based work item types: Edit work items in this node permission set to Allow under the corresponding Area Path. - To modify test plan properties such as build and test settings: Manage test plans permission set to Allow under the corresponding Area Path. - To create and delete test suites, add and remove test cases from test suites, or change test configurations associated with test suites: Manage test suites permission set to Allow under the corresponding Area Path. - Project-level permissions that you might need include: - To view, create, or delete test runs: View test runs, Create test runs, and Delete test runs permission set to Allow. - To manage test configurations or test environments: Manage test configurations and Manage test environments. - To add new tags to test-based work items: Create tag definition permission set to Allow. - To move test-based work items out of a project: Move work items out of this project permission set to Allow. - To delete or restore test-based work items: Delete and restore work items permission set to Allow. - To permanently delete test-based work items: Permanently delete work items permission set to Allow. |
For more information, see Manual test access and permissions.
How to view the default failure types
Failure type categorization is part of the post-test run experience in Azure Test Plans. This feature is only applicable to test cases marked as Failed. To mark a test failure using the default failure type, follow the below steps:
Navigate to Azure Test Plans from your Azure DevOps project. Open a Test suite containing test cases or create a new test case inside a suite.
Navigate to Execute tab of the given test suite.
Mark a test case as Failed, which creates a new test run for against the test case.
Select Runs from the Test Plans menu. Select the respective test run from the list shown.
Run Summary tab is shown by default. Select on Test results tab.
Select on Update Analysis and a dialog box appears. Users can choose the failure type for the test case from the dropdown menu. Refer to the screenshot for the failure type values shown by default.
Add custom values for Failure Type
A new set of REST APIs is provided for seamless management of failure types in Azure Test Plans. Users can add new failure types, delete existing ones and view the list of failure types for a given project. Follow the instructions provided in the following links to explore the various APIs.