Running .orderedlist file using TFS test agent (on a remote machine) ?

Pavel Malichenko 1 Reputation point
2020-09-15T22:34:21.47+00:00

Hi,

I was wondering if it is possible to run CodedUI tests using TFS test agent (on a remote machine) in a specific order, using the option of creating an ordered list (.orderedlist extension) of required tests ?

Thanks.

Visual Studio Testing
Visual Studio Testing
Visual Studio: A family of Microsoft suites of integrated development tools for building applications for Windows, the web and mobile devices.Testing: The act or process of applying a test as a means of analysis or diagnosis.
253 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Tianyu Sun-MSFT 13,091 Reputation points Microsoft Employee
    2020-09-16T08:45:54.093+00:00

    Hello Pavel,

    Do you mean .orderedtest file? If you use MSTest framework (v1), you can try to use .orderedtest file to run tests in order, please check this document: How to: Create an Ordered Test. If you use MSTest framework (v2), I’m afraid, MSTest framework v2 does not support ordered tests.

    For MSTest framework v1, since MSTest.exe works on MSTest framework (v1), you can try to set the testsettings file(for MSTest framework v1. Please also note, testsettings file is replaced by runsettings file) and use “MSTest.exe” command lines to run tests in order by using test agents.

    For example mstest.exe /testcontainer:”XXXX\XXXX\XX.orderedtest /testsettings:XXXXX.Testsettings”.

    For MSTest framework v2, MSTest v2 doesn’t support ordered test currently. This is a related feature request(or issue report): MSTestv2 does not support ordered Tests, which status is “Open”, I suggest you can add comments and follow this thread.

    By the way, it is recommended that you should consider to make the tests independent of order and data.

    Sincerely,
    Tianyu

    • If the answer is helpful, please click "Accept Answer" and upvote it.
      Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.
    No comments