Preparing Appium Tests for Upload
Important
Visual Studio App Center is scheduled for retirement on March 31, 2025. While you can continue to use Visual Studio App Center until it is fully retired, there are several recommended alternatives that you may consider migrating to.
The steps to prepare an app and its test suite for upload vary depending on the test framework. This guide how to prepare Appium tests using Java with JUnit for upload to App Center. For guidance on authoring Appium tests, see the Appium documentation.
Note the following limitations for Appium support:
Note
WebView context, Chrome Web Driver and Chrome browser testing with browserName
capability are supported!
- No support for TestNG (only JUnit tests are supported).
- No support for Android 4.2 or prior. No support for deprecated UIAutomator driver.
- No support for iOS 9.2.1 or prior. No support for deprecated UIAutomation iOS driver.
- No support for JUnit @Category attribute. (Can use Include/Exclude instead)
- Maven version must be at least 3.3.9.
- Current Appium version is 1.22.0. It is regulary updated with new releases.
- JUnit 4.9 - 4.12 is supported; we don't support JUnit 5.
- Tests must target precisely one app. (
MobileCapabilityType.FULL_RESET
is supported)
Note
In some cases, tests can still work in App Center if using unsupported tooling or features. However, that unsupported functionality isn't QA'd in future updates and could break without warning.
Prerequisites
Tests will be run using Maven Surefire, which requires tests to follow certain naming conventions:
"**/Test*.java" - includes all of its subdirectories and all Java filenames that start with "Test".
"**/*Test.java" - includes all of its subdirectories and all Java filenames that end with "Test".
"**/*Tests.java" - includes all of its subdirectories and all Java filenames that end with "Tests".
"**/*TestCase.java" - includes all of its subdirectories and all Java filenames that end with "TestCase".
Before attempting to upload to App Center Test, make sure that running tests locally on your machine using Maven works:
➜ AppiumTest git:(main) ✗ mvn verify
...
Running MainTest
started: SimpleTest (MainTest)
Setting up capabilities
failed
finished
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.728 sec <<< FAILURE!
SimpleTest(MainTest) Time elapsed: 0.594 sec <<< ERROR!
...
If you're unable to run the tests using command line locally, tests will also not work in App Center Test.
1. Changes to the build system
Step 1 - Add dependency
You'll need to add a dependency for the Appium test extensions:
<dependency>
<groupId>com.microsoft.appcenter</groupId>
<artifactId>appium-test-extension</artifactId>
<version>1.6</version>
</dependency>
This code will ensure the enhanced Android and iOS drivers are available at compile time. The enhanced drivers are provided primarily to enable the label
feature. See Step 4 for more detail on the label
feature.
Step 2 - Add upload profile
Copy this snippet into your pom.xml
in the <profiles>
tag. If there's no <profiles>
section in your pom, make one.
The profile, when activated, will pack your test classes and all dependencies into the target/upload
folder, ready to be uploaded to Test.
2. Changes to the tests
Step 1 - Add imports
Import these packages into your test classes:
import com.microsoft.appcenter.appium.Factory;
import com.microsoft.appcenter.appium.EnhancedAndroidDriver;
import org.junit.rules.TestWatcher;
import org.junit.Rule;
Step 2 - Instantiate the TestWatcher
Insert this declaration in each of your test classes:
@Rule
public TestWatcher watcher = Factory.createWatcher();
Step 3 - Update your driver declaration
Replace your declaration of AndroidDriver<MobileElement>
with EnhancedAndroidDriver<MobileElement>
or IOSDriver<MobileElement>
with EnhancedIOSDriver<MobileElement>
private static EnhancedAndroidDriver<MobileElement> driver;
Step 4 - Update your driver instantiations
Replace the way you instantiate your driver, such that lines in the form of:
driver = new AndroidDriver<MobileElement>(url, capabilities);
...are changed to:
driver = Factory.createAndroidDriver(url, capabilities);
Using these drivers will still allow you to run your tests locally without additional modifications, but enables you to "label" test steps in your test execution using driver.label("text")
. The text and a screenshot from the device will be visible in test report in App Center.
It's recommended to call driver.label
in the @After
method, which takes a screenshot of the app final state. An example @After
method for a test could look like this code:
@After
public void TearDown(){
driver.label("Stopping App");
driver.quit();
}
3. Upload to App Center Test
Steps to upload a test:
Generate an App Center Test upload command using the instructions at starting a test run.
Pack your test classes and all dependencies into the
target/upload
folder:mvn -DskipTests -P prepare-for-upload package
Run the upload command:
appcenter test run appium --app "APP_ID" --devices "DEVICE_SET_ID" --app-path PATH_TO_FILE.apk --test-series "main" --locale "en_US" --build-dir target/upload
4. Performance Troubleshooting
Tests on devices in App Center will execute slightly slower than on a local device. Normally, slower execution is outweighed by having more devices available, allowing parallel test runs.
There are three main sources of slower test runs: re-signing, reinstallation, and network tasks.
Re-signing (on iOS)
Before being installed on the iOS device, your app goes through a process called re-signing. This process is necessary to make the provisioning profile match the device in the cloud. Re-signing does take some time, typically ~1-2 minutes. Rarely, re-signing also causes performance degradation because re-signed apps are cached. The time consuming process will only run once per binary.
If your Continuous Delivery setup updates the IPA version before building and testing, then the binary will be different for each test and the re-signing penalty will occur more often.
Reinstallation
On a shared device cloud, it's important for us to guarantee that devices are cleaned between each test. The next customer using the device may be someone from another organization. In App Center Test, the app is automatically uninstalled after the completion of your test run.
It's possible to omit MobileCapabilityType.FULL_RESET
and set MobileCapabilityType.NO_RESET
to true
to speed up test execution. See Reset Strategies for details.
Network Tasks
Local network tasks are faster because the server is closer and more dedicated to the remote host.