Run UI Tests using App Center REST API
Important
Visual Studio App Center is scheduled for retirement on March 31, 2025. While you can continue to use Visual Studio App Center until it is fully retired, there are several recommended alternatives that you may consider migrating to.
This example shows how to upload an app package, test binaries, and queue UI tests using our API.
- Use the create test run POST method to receive a test run ID, which you'll use later for the Response Header.
Sample Request https://api.appcenter.ms/v0.1/apps/MSKASANI/DroidAppSampleXamarin/test_runs
- Create file hashes for each file you want to upload to App Center Test.
You can get the file hashes by running the command shasum
. This utility is available in Mac/Linux machines by default. For Windows, consider installing Git Bash and use it instead.
Example
ankasani@AK--PC MINGW64 ~/source/repos/AppXamarin/sampleapp-xamarin-master/UITest/bin/Debug/testdlls (master)
$ shasum nunit.framework.dll
a2d0dbc920fb3790d4d625e5c47be042ca32c051 *nunit.framework.dll
$ shasum Xamarin.UITest.dll
c1f7342d05a9d95580a507156207e6f9283a2c8d *Xamarin.UITest.dll
$ shasum UITest.dll
542153902bc889d0d85a83b005854b5ebd9864f1 *UITest.dll
$ shasum com.mobilecenter.sampleappxamarin.apk
3f3677f8db9ba7c07ac77d4f01f912e518a6ca16 *com.mobilecenter.sampleappxamarin.apk
- Create a store for files using POST and hashes value calculated from the previous step.
Sample Request
https://api.appcenter.ms/v0.1/apps/MSKASANI/DroidAppSampleXamarin/test_runs/fb0cc373-aa4a-455f-b083-4fc1afa87f2e/hashes/batch
file_type - app-file for apk\ipa and test-file for test assemblies/test related files.
checksum - hash values from step 2
relative_path - you can define File Name with Extension, no need to specify the complete path.
The JSON Body looks like:
[
{
"file_type":"app-file",
"checksum":"3f3677f8db9ba7c07ac77d4f01f912e518a6ca16",
"relative_path":"com.mobilecenter.sampleappxamarin.apk"
},
{
"file_type":"test-file",
"checksum":"a2d0dbc920fb3790d4d625e5c47be042ca32c051",
"relative_path":"nunit.framework.dll"
},
{
"file_type":"test-file",
"checksum":"c1f7342d05a9d95580a507156207e6f9283a2c8d",
"relative_path":"Xamarin.UITest.dll"
},
{
"file_type":"test-file",
"checksum":"542153902bc889d0d85a83b005854b5ebd9864f1",
"relative_path":"UITest.dll"
}
]
The response looks like:
[
{
"fileType": "app-file",
"checksum": "3f3677f8db9ba7c07ac77d4f01f912e518a6ca16",
"relativePath": "com.mobilecenter.sampleappxamarin.apk",
"uploadStatus":
{
"statusCode": 412,
"location": "https://testcloud.xamarin.com/v0.1/direct_uploads?token=ZXlKMGVYQWlPaUpLVjFRaUxDSmhiR2NpT2lKU1V6STFOaUo5LmV5SjBaWE4w%0AWDNKMWJsOXBaQ0k2SWpjNU5EVmpNbUZtTFRFeFpXTXROR1ptWVMwNU1XSTBM%0AVFExTUdJd09EVTNNbU5sT1NJc0ltVjRjQ0k2TVRVek16Z3dOVFEzTUgwLlJO%0AaTQ3VTNPYTB5YWswMVRVdTV6Q0dfWFhwV25QUWJDdkJaNmVCRWRmOENuQ19U%0AbmdpRGI4MU0yYk1lTlZNY1VCVmRWLXlkRjh4dElLaU1yNFdtQUFhOFdoay02%0AOVVjb29rYkNTYUU2eUlzamowZlZPZVJhR2lFdGpvLWJHTFJLcXNGY0ZTeUdR%0ARTQzQUZ1TFlyTWlCRnowMElpd3ZudjYzZk9wQnRGaXhNa2lNNjhUTWxxVnNa%0AOFl3TFlTeU5KSWFvYVJCM0ZZTUkwajNYOWdVb3VJclVjNnpqcGNkYkxCcWJ0%0AVUpyN0ZFaTdJeWY4UG9rMXZxTzRlUExiaEszTmItTmd0Q2JuckZzS2x6Z3Za%0AaktUWHZZb2oya0FGWXZHaVZ4LVhfNVhUcGdiVnE1Z2dxVEtIdjRaUGFuaGR4%0AdnFvZ2YxRVYwWUJYaG40YUJLM3BnUVBodw%3D%3D%0A"
}
},
{
"fileType": "test-file",
"checksum": "a2d0dbc920fb3790d4d625e5c47be042ca32c051",
"relativePath": "nunit.framework.dll",
"uploadStatus":
{
"statusCode": 412,
"location": "https://testcloud.xamarin.com/v0.1/direct_uploads?token=ZXlKMGVYQWlPaUpLVjFRaUxDSmhiR2NpT2lKU1V6STFOaUo5LmV5SjBaWE4w%0AWDNKMWJsOXBaQ0k2SWpjNU5EVmpNbUZtTFRFeFpXTXROR1ptWVMwNU1XSTBM%0AVFExTUdJd09EVTNNbU5sT1NJc0ltVjRjQ0k2TVRVek16Z3dOVFEzTUgwLlJO%0AaTQ3VTNPYTB5YWswMVRVdTV6Q0dfWFhwV25QUWJDdkJaNmVCRWRmOENuQ19U%0AbmdpRGI4MU0yYk1lTlZNY1VCVmRWLXlkRjh4dElLaU1yNFdtQUFhOFdoay02%0AOVVjb29rYkNTYUU2eUlzamowZlZPZVJhR2lFdGpvLWJHTFJLcXNGY0ZTeUdR%0ARTQzQUZ1TFlyTWlCRnowMElpd3ZudjYzZk9wQnRGaXhNa2lNNjhUTWxxVnNa%0AOFl3TFlTeU5KSWFvYVJCM0ZZTUkwajNYOWdVb3VJclVjNnpqcGNkYkxCcWJ0%0AVUpyN0ZFaTdJeWY4UG9rMXZxTzRlUExiaEszTmItTmd0Q2JuckZzS2x6Z3Za%0AaktUWHZZb2oya0FGWXZHaVZ4LVhfNVhUcGdiVnE1Z2dxVEtIdjRaUGFuaGR4%0AdnFvZ2YxRVYwWUJYaG40YUJLM3BnUVBodw%3D%3D%0A"
}
},
{
"fileType": "test-file",
"checksum": "c1f7342d05a9d95580a507156207e6f9283a2c8d",
"relativePath": "Xamarin.UITest.dll",
"uploadStatus":
{
"statusCode": 412,
"location": "https://testcloud.xamarin.com/v0.1/direct_uploads?token=ZXlKMGVYQWlPaUpLVjFRaUxDSmhiR2NpT2lKU1V6STFOaUo5LmV5SjBaWE4w%0AWDNKMWJsOXBaQ0k2SWpjNU5EVmpNbUZtTFRFeFpXTXROR1ptWVMwNU1XSTBM%0AVFExTUdJd09EVTNNbU5sT1NJc0ltVjRjQ0k2TVRVek16Z3dOVFEzTUgwLlJO%0AaTQ3VTNPYTB5YWswMVRVdTV6Q0dfWFhwV25QUWJDdkJaNmVCRWRmOENuQ19U%0AbmdpRGI4MU0yYk1lTlZNY1VCVmRWLXlkRjh4dElLaU1yNFdtQUFhOFdoay02%0AOVVjb29rYkNTYUU2eUlzamowZlZPZVJhR2lFdGpvLWJHTFJLcXNGY0ZTeUdR%0ARTQzQUZ1TFlyTWlCRnowMElpd3ZudjYzZk9wQnRGaXhNa2lNNjhUTWxxVnNa%0AOFl3TFlTeU5KSWFvYVJCM0ZZTUkwajNYOWdVb3VJclVjNnpqcGNkYkxCcWJ0%0AVUpyN0ZFaTdJeWY4UG9rMXZxTzRlUExiaEszTmItTmd0Q2JuckZzS2x6Z3Za%0AaktUWHZZb2oya0FGWXZHaVZ4LVhfNVhUcGdiVnE1Z2dxVEtIdjRaUGFuaGR4%0AdnFvZ2YxRVYwWUJYaG40YUJLM3BnUVBodw%3D%3D%0A"
}
},
{
"fileType": "test-file",
"checksum": "542153902bc889d0d85a83b005854b5ebd9864f1",
"relativePath": "UITest.dll",
"uploadStatus":
{
"statusCode": 412,
"location": "https://testcloud.xamarin.com/v0.1/direct_uploads?token=ZXlKMGVYQWlPaUpLVjFRaUxDSmhiR2NpT2lKU1V6STFOaUo5LmV5SjBaWE4w%0AWDNKMWJsOXBaQ0k2SWpjNU5EVmpNbUZtTFRFeFpXTXROR1ptWVMwNU1XSTBM%0AVFExTUdJd09EVTNNbU5sT1NJc0ltVjRjQ0k2TVRVek16Z3dOVFEzTUgwLlJO%0AaTQ3VTNPYTB5YWswMVRVdTV6Q0dfWFhwV25QUWJDdkJaNmVCRWRmOENuQ19U%0AbmdpRGI4MU0yYk1lTlZNY1VCVmRWLXlkRjh4dElLaU1yNFdtQUFhOFdoay02%0AOVVjb29rYkNTYUU2eUlzamowZlZPZVJhR2lFdGpvLWJHTFJLcXNGY0ZTeUdR%0ARTQzQUZ1TFlyTWlCRnowMElpd3ZudjYzZk9wQnRGaXhNa2lNNjhUTWxxVnNa%0AOFl3TFlTeU5KSWFvYVJCM0ZZTUkwajNYOWdVb3VJclVjNnpqcGNkYkxCcWJ0%0AVUpyN0ZFaTdJeWY4UG9rMXZxTzRlUExiaEszTmItTmd0Q2JuckZzS2x6Z3Za%0AaktUWHZZb2oya0FGWXZHaVZ4LVhfNVhUcGdiVnE1Z2dxVEtIdjRaUGFuaGR4%0AdnFvZ2YxRVYwWUJYaG40YUJLM3BnUVBodw%3D%3D%0A"
}
}
]
Now we need to upload the actual files. From the previous response body, identify the
location
that contains the URL of the files you're uploading.- In the Headers, you need to pass Content-Type: multipart/form-data
- In the Body, you need to use the type Body_Form with the values:
relative_path
andfile_type
: from the previous Response Bodyfile
: Path to the actual file
You're on the right track if you get 201 Created
as the response like below.
(We're halfway there)
Repeat the previous step for the rest of the files you want to upload.
We're ready to Execute the test now. For this, we need a device_slug.
Note: You should first create "Device Sets" in your Org or you can use the REST API to create device sets.
Then, you can use the GET Method to get the list of device sets.
Sample Request: https://api.appcenter.ms/v0.1/apps/MSKASANI/DroidAppSampleXamarin/owner/device_sets
Corresponding Response Body:
{
"id": "00917fb6-f30f-4d36-bd53-f900a94efe9f",
"name": "UI Test",
"slug": "ui-test",
"osVersionCount": 1,
"manufacturerCount": 1,
"owner":
{
"type": "organization",
"id": "5dd75115-3832-4f10-9cfe-5ac8cc8a51a5",
"displayName": "MSKASANI",
"name": "MSKASANI"
},
"deviceConfigurations":
{
{
"id": "717cfc38-4817-41fc-8a8e-06beef4f73b2",
"image":
{
"thumb": "https://testcloud-prod-system-files.s3-eu-west-1.amazonaws.com/system_files/0dee6ee7-6839-4e4e-ad0b-333e3952f75c?response-cache-control=max-age%3D157788000&AWSAccessKeyId=AKIAI4UZT4FCOF2OTJYQ&Signature=r9EEm/x0YAA4NhRJ%2BhVSQwGtBbI%3D&Expires=1691079085"
},
"os": "8.1.0",
"osName": "Android 8.1.0",
"model":
{
"name": "Google Pixel 2 XL",
"manufacturer": "Google",
"releaseDate": "October 2017",
"formFactor": "phone"
}
}
}
}
- We're now ready to use the Test start API to trigger the test! The Request body below is created for Xamarin.UITest Framework. If you're using other frameworks, refer to the end of the doc for more details!
Sample JSON Request Body:
{
"test_framework":"ui_test",
"device_selection":"MSKASANI/ui-test",
"locale":"en_US",
"test_series":"apimain",
"test_parameters":
{
"tests":
{
"method":
{
"UITest.dll":
[
"UITest.Tests(Android).AppLaunches",
"UITest.Tests(iOS).AppLaunches"
]
},
"fixture":
{
"UITest.dll":
[
"UITest.Tests(Android)",
"UITest.Tests(iOS)"
]
}
}
}
}
Corresponding response body:
{
"accepted_devices":["Google Pixel 2 XL (8.1.0)"],
"rejected_devices":[]
}
If you now go to the Test page of your App, you'll see...
Congratulations! :)
PS: For other Test Frameworks, if you want to know how the JSON looks like (for instance, Appium test)
Run the Test command manually with --debug
switch.
appcenter test run appium --app "MSKASANI/Appium" --devices "MSKASANI/my" --app-path C:\VSAC\AllTest\AppCenter-Test-Samples-main\Appium\Android\swiftnote.apk --test-series "main" --locale "en_US" --build-dir C:\VSAC\AllTest\AppCenter-Test-Samples-main\Appium\Android\Mavenex\target\upload --debug
Once the test is complete, you can check out the Command output to see how the Request Body looks like:
BONUS! This way you can use the CLI output to see what how our commands queue the Tests using our API!