Running .orderedlist file using TFS test agent (on a remote machine) ?

Question

Hi,

I was wondering if it is possible to run CodedUI tests using TFS test agent (on a remote machine) in a specific order, using the option of creating an ordered list (.orderedlist extension) of required tests ?

Thanks.

Answer 1

Hello Pavel,

Do you mean .orderedtest file? If you use MSTest framework (v1), you can try to use .orderedtest file to run tests in order, please check this document: How to: Create an Ordered Test. If you use MSTest framework (v2), I’m afraid, MSTest framework v2 does not support ordered tests.

For MSTest framework v1, since MSTest.exe works on MSTest framework (v1), you can try to set the testsettings file(for MSTest framework v1. Please also note, testsettings file is replaced by runsettings file) and use “MSTest.exe” command lines to run tests in order by using test agents.

For example mstest.exe /testcontainer:”XXXX\XXXX\XX.orderedtest /testsettings:XXXXX.Testsettings”.

For MSTest framework v2, MSTest v2 doesn’t support ordered test currently. This is a related feature request(or issue report): MSTestv2 does not support ordered Tests, which status is “Open”, I suggest you can add comments and follow this thread.

By the way, it is recommended that you should consider to make the tests independent of order and data.

Sincerely,
Tianyu

• If the answer is helpful, please click "Accept Answer" and upvote it.
  Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.