Running .orderedlist file using TFS test agent (on a remote machine) ?

Pavel Malichenko 1 Reputation point
2020-09-15T22:34:21.47+00:00

Hi,

I was wondering if it is possible to run CodedUI tests using TFS test agent (on a remote machine) in a specific order, using the option of creating an ordered list (.orderedlist extension) of required tests ?

Thanks.

Visual Studio Testing
Visual Studio Testing
Visual Studio: A family of Microsoft suites of integrated development tools for building applications for Windows, the web and mobile devices.Testing: The act or process of applying tests as a means of analysis or diagnosis.
329 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Tianyu Sun-MSFT 27,356 Reputation points Microsoft Vendor
    2020-09-16T08:45:54.093+00:00

    Hello Pavel,

    Do you mean .orderedtest file? If you use MSTest framework (v1), you can try to use .orderedtest file to run tests in order, please check this document: How to: Create an Ordered Test. If you use MSTest framework (v2), I’m afraid, MSTest framework v2 does not support ordered tests.

    For MSTest framework v1, since MSTest.exe works on MSTest framework (v1), you can try to set the testsettings file(for MSTest framework v1. Please also note, testsettings file is replaced by runsettings file) and use “MSTest.exe” command lines to run tests in order by using test agents.

    For example mstest.exe /testcontainer:”XXXX\XXXX\XX.orderedtest /testsettings:XXXXX.Testsettings”.

    For MSTest framework v2, MSTest v2 doesn’t support ordered test currently. This is a related feature request(or issue report): MSTestv2 does not support ordered Tests, which status is “Open”, I suggest you can add comments and follow this thread.

    By the way, it is recommended that you should consider to make the tests independent of order and data.

    Sincerely,
    Tianyu

    • If the answer is helpful, please click "Accept Answer" and upvote it.
      Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.
    0 comments