Xamarin.iOS UITest does not run with category

Question

Hi,

I have a lot of uitests for Xamarin.Forms app. These tests under categories. (For example LoginTest category has 10 tests, InboxTest category has 40 tests etc.) When I select 1 category (or when I select more than categories) and run it on iOS simulator, firstly tests of these categories runs, but after a while some tests don't run.

And I encounter this error:

Connection closed
Stack trace:
at MonoDevelop.Core.Execution.RemoteProcessConnection.SendMessage[RT] (MonoDevelop.Core.Execution.BinaryMessage1[RT] message) [0x00024] in /Users/builder/azdo/_work/1/s/main/src/core/MonoDevelop.Core/MonoDevelop.Core.Execution/RemoteProcessConnection.cs:374 at MonoDevelop.UnitTesting.NUnit.External.ExternalTestRunner.Run (MonoDevelop.UnitTesting.NUnit.External.IRemoteEventListener listener, System.String[] nameFilter, System.String path, System.String suiteName, System.Collections.Generic.List1[T] supportAssemblies, System.String testRunnerType, System.String testRunnerAssembly, System.String crashLogFile) [0x00099] in /Users/builder/azdo/_work/1/s/main/src/addins/MonoDevelop.UnitTesting.NUnit/MonoDevelop.UnitTesting.NUnit/ExternalTestRunner.cs:91
at Microsoft.VisualStudio.Threading.JoinableTask.CompleteOnCurrentThread () [0x001b6] in C:\A_work\107\s\src\Microsoft.VisualStudio.Threading\JoinableTask.cs:916
at Microsoft.VisualStudio.Threading.JoinableTask1[T].CompleteOnCurrentThread () [0x00000] in C:\A\_work\107\s\src\Microsoft.VisualStudio.Threading\JoinableTask1.cs:84
at Microsoft.VisualStudio.Threading.JoinableTaskFactory.Run[T] (System.Func1[TResult] asyncMethod, Microsoft.VisualStudio.Threading.JoinableTaskCreationOptions creationOptions) [0x0000e] in C:\A\_work\107\s\src\Microsoft.VisualStudio.Threading\JoinableTaskFactory.cs:505 at Microsoft.VisualStudio.Threading.JoinableTaskFactory.Run[T] (System.Func1[TResult] asyncMethod) [0x00000] in C:\A_work\107\s\src\Microsoft.VisualStudio.Threading\JoinableTaskFactory.cs:484
at MonoDevelop.UnitTesting.NUnit.NUnitAssemblyTestSuite.RunUnitTest (MonoDevelop.UnitTesting.UnitTest test, System.String suiteName, System.String pathName, System.String testName, MonoDevelop.UnitTesting.TestContext testContext) [0x00265] in /Users/builder/azdo/_work/1/s/main/src/addins/MonoDevelop.UnitTesting.NUnit/MonoDevelop.UnitTesting.NUnit/NUnitAssemblyTestSuite.cs:535
at MonoDevelop.UnitTesting.NUnit.NUnitTestSuite.OnRun (MonoDevelop.UnitTesting.TestContext testContext) [0x00000] in /Users/builder/azdo/_work/1/s/main/src/addins/MonoDevelop.UnitTesting.NUnit/MonoDevelop.UnitTesting.NUnit/NUnitTestSuite.cs:79
at MonoDevelop.UnitTesting.UnitTest.Run (MonoDevelop.UnitTesting.TestContext testContext) [0x0001c] in /Users/builder/azdo/_work/1/s/main/src/addins/MonoDevelop.UnitTesting/Services/UnitTest.cs:351

When I run it one by one, they run.
My Visual Studio mac version is 8.8.4 (up to date)
Xamarin.TestCloud.Agent version is 0.22.1
Xamarin.UITest version is 3.0.12

What must I do? Could someone help me please?
Thank you in advance.

Answer 1

Hi @JarvanZhang ,thank you for reply.
It works same with real phone. In addition uitests on real phones work too slow.
I set categories with this way:

public enum EGeneralTestOrder  
    {  
        DeleteAllTestFile = 1,  
        LoginTest,  
        MyFolderTest  
    }  


public enum LoginTestOrder  
    {  
        Login_Success = 1,  
        Login_EmptyUsername_Unsuccess,  
        Login_EmptyPassword_Unsuccess  
    }  

public abstract class BaseTest  
    {     
        protected const int RepeatCount = 1;  
        protected const string LoginTestPrefix = "LoginTest";  
        protected const string MyFolderTestPrefix = "MyFolderTest";  
        //...  
    }  

[TestFixture, Order((int)(EGeneralTestOrder.LoginTest)), Category("LoginTests")]  
    public class LoginTest : BaseTest  
    {  
        public LoginTest() : base()  
        {  
  
        }  
  
        [Category(LoginTestPrefix)]  
        [TestCase(TestConfigurations.BaseURL, Username1, Password1), Order((int)(LoginTestOrder.Login_Success))]  
        [Retry(RepeatCount)]  
        public void Login_Success (string url, string username, string password)  
        {  
            try  
            {  
                //test codes...  
            }  
            catch (Exception ex)  
            {  
                Assert.Fail(ex.ToString());  
            }  
        }  
  
        [Category(LoginTestPrefix)]  
        [TestCase(TestConfigurations.BaseURL, "", Password1), Order((int)(LoginTestOrder.Login_EmptyUsername_Unsuccess))]  
        [Retry(RepeatCount)]  
        public void Login_EmptyUsername_Unsuccess (string url, string username, string password)  
        {  
            try  
            {  
                //test codes...  
            }  
            catch (Exception ex)  
            {  
                Assert.Fail(ex.ToString());  
            }  
        }  
    }  

I have been running uitests for a long time (about 1 year) That's why I don't think issue is category.
But still I added how I set categories to see if I have any errors setting category.