Azure DevOps: JUnit Test Results group by stage not possible

Question

Hi everyone,

We are using Synapse Pipelines to process data, and we have implemented unit tests in Python using JUnit to test them in Azure DevOps pipelines.

We discovered, it is not possible to group the test results by environment:

Here's an example:

• Test ran in both stages
• Having a look at the details, all tests are grouped in a single bucket and you cannot tell between stages.
• Every environment executes 412 tests, but only the sum is reported.

Is there any way to overcome this? 

Maybe organizing the pipelines in a different way?

Answer 1

Hello Paul Hernandez

It seems that you are trying to group JUnit test results by environment in Azure DevOps pipelines. Unfortunately, grouping test results by environment is not currently possible in Azure DevOps pipelines.

From this document:

During test execution, a test might spawn multiple instances or tests that contribute to the overall outcome. Some examples are, tests that are rerun, tests composed of an ordered combination of other tests (ordered tests) or tests having different instances based on an input parameter (data driven tests).

As these tests are related, they must be reported together with the overall outcome derived from the individual instances or tests. These test results are reported as a summarized test result in the Tests tab:

To see test results of individual tests in the ordered test suite you may need to download the *.trx file and open it in Visual Studio

A similar thread has been discussed here: https://stackoverflow.com/questions/60397891/test-results-for-ordered-test-are-grouped-in-azure-devops-pipeline

However, a workaround has been discussed in the above thread:

<Copied the below from the stackoverflow>

All the tests I wanted to run were in a single ordered test. I changed the build definition to run them individually and not as a part of a single ordered test. The only drawback of following this approach is we cannot rerun the failed test

I hope this helps! Let me know if you have any further questions.