Build Pipeline, Code Coverage - internal server error
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(259.20000000000005, 60%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Marek Łamasz
0
Reputation points
Hello,
I've came across an unusual issue while trying to enable Code coverage for pull requests.
I've enabled codeCoverageEnabled in the VSTest@3 task in my build pipeline and set this pipeline as a Build Validation in target branch policy settings.
After creating a Pull Request, the build validation pipeline executes successfully, but there is no status check sent to the PR and after switching to "Code Coverage" tab in the pipeline run details, the following error can be seen:
There is an internal server error. The code coverage job has failed.
There are no errors in the pipelines logs. All the steps execute properly, unit tests are run and the test report can be seen in the "Tests" tab.
Results File: D:\a\_temp\TestResults\VssAdministrator_fv-az902-812_2025-03-10_16_56_35.trx
Attachments:
D:\a\_temp\TestResults\17e9a4fb-d181-4802-9954-e2f35847ecb3\VssAdministrator_fv-az902-812_2025-03-10.16_57_58.coverage
Test Run Successful.
Total tests: 2403
Passed: 2398
Skipped: 5
Total time: 2.1072 Minutes
Vstest.console.exe exited with code 0.
**************** Completed test execution *********************
Test results files: D:\a\_temp\TestResults\VssAdministrator_fv-az902-812_2025-03-10_16_56_35.trx
Created test run: 1027454
Publishing test results: 2403
Publishing test results to test run '1027454'.
TestResults To Publish 2403, Test run id:1027454
Test results publishing 1000, remaining: 1403. Test run id: 1027454
Test results publishing 1000, remaining: 403. Test run id: 1027454
Test results publishing 403, remaining: 0. Test run id: 1027454
Published test results: 2403
Publishing Attachments: 2
Completed TestExecution Model...
Finishing: Run Unit Tests
The thing is - when running the same pipeline outside the Pull Request, simply by using "Run pipeline" button and choosing the same branch as a resource, the pipeline runs fine and "Code Coverage" tab displays properly, along with the coverage report generated. So the jobs themselves work fine and the coverage report generates fine too.
As there are no errors/warnings in the pipeline logs, I have no idea how to debug and get more information what the actual error is. I suspect it fails in some internal logic when it tries to set PR status...
Azure DevOps
{count} votes
-
Arko 4,165 Reputation points Microsoft External Staff Moderator2025-03-11T11:06:48.08+00:00 Hi Marek Łamasz, could you kindly check if your pipeline has necessary permissions to publish code coverage results when running as a PR build validation. Could you kindly check if "Allow scripts to access the OAuth token" setting is missing?
Also, kindly check under Build validation if the correct pipeline is selected / the "Report build status" option is enabled / "Automatically link work items" option is enabled.MS Guide for troubleshooting Code coverage- https://learn.microsoft.com/en-us/troubleshoot/developer/visualstudio/ide/troubleshooting-code-coverage
-
Marek Łamasz 0 Reputation points
2025-03-11T12:53:06.4766667+00:00 Hi, thank you for you answer.
I've read about "Allow scripts to access the OAuth token" option and I believe it's for a "classic" pipelines. We use YAML pipelines. Pipelines do run on
Azure-hostedagent.necessary permissions to publish code coverage results
Not really sure, where in DevOps should I look for those permissions to set.
In the past, I has to add explicit permissions for
Project Collection Build ServiceandProject Collection Build Service Accountsbut this was in order to allow pipelines access to our Azure Artifact Feedcheck under Build validation if the correct pipeline is selected
It is. It has to be, since that error indicates it tries to perform Status Check on pull requests but encounters an unknown error.
the "Report build status" option is enabled
I believe this is also an option from classic pipelines.
-
Marek Łamasz 0 Reputation points
2025-03-11T15:50:46.41+00:00 check under Build validation if the correct pipeline is selected
It is. It has to be, since that error occurs only when the pipeline is triggered by the pull request. So it indicates it maybe tries to send Status Check on pull requests but encounters an unknown error...
the "Report build status" option is enabled
I believe this is also an option from classic pipelines.
-
Ryan Hill 30,326 Reputation points Microsoft Employee Moderator2025-03-17T21:15:13.1833333+00:00 @Marek Łamasz I'm just curious, do you get the same behavior if you downgrade the
@VSTesttask? -
Marek Łamasz 0 Reputation points
2025-03-18T20:20:37.62+00:00 @Ryan Hill @Arko I've tested switching
@VsTest3to@VsTest2- same issue.What's more - I've wanted to try publishing the coverage results explicitly, to see if maybe I can get some error message out of it.
I've checked the logs of
@VsTest3task withcodeCoverageEnabledand checked the path ofVssAdministrator_fv-az567-577_2025-03-18.18_28_08.coveragefile that is being generated.Then, I've added additional task -
PublishCodeCoverageResults@2- withsummaryFileLocationpointing to that.coveragefile. The result is, in the logs ofPublishCodeCoverageResultstask I see the following line:It seems that the report file 'D:\a\_temp\TestResults\3f416982-a43a-4945-a633-c6888988a81b\VssAdministrator_fv-az618-761_2025-03-18.18_41_29.coverage' is a binary format generated by a Visual Studio code coverage tool. Please convert to XML format with 'CodeCoverage.exe' (See: https://github.com/danielpalme/ReportGenerator/wiki/Visual-Studio-Coverage-Tools#codecoverageexe)Which is confusing, as in Code coverage for pull requests we can read that:
the code coverage for pull requests feature discussed in this document is currently available only for .NET and .NET core projects using the Visual Studio code coverage results format (file extension .coverage)
so I don't know if the default output format of the
@VsTestis the problem...Still, the biggest issue still remains the error message
There is an internal server error. The code coverage job has failed.which I have no idea how to debug or how to get any indication of what the actual internal error is
Googling that error message I've found literally nothing, so I assume it's not a common problem 🙂
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 79%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
Suhaib Al-Yassin 0 Reputation points2025-04-17T12:15:05.4566667+00:00 Hello,
Have you been able to identify a solution for this issue? we're currently experiencing same issue.
Sign in to comment
Sign in to answer