Load Test Run - Create Or Update Test Run
Create and start a new test run with the given test run Id.
PATCH https://{endpoint}/test-runs/{testRunId}?api-version=2022-11-01
PATCH https://{endpoint}/test-runs/{testRunId}?api-version=2022-11-01&oldTestRunId={oldTestRunId}
URI Parameters
| Name | In | Required | Type | Description |
|---|---|---|---|---|
|
endpoint
|
path | True |
string |
|
|
test
|
path | True |
string minLength: 2maxLength: 50 pattern: ^[a-z0-9_-]*$ |
Unique test run identifier for the load test run, must contain only lower-case alphabetic, numeric, underscore or hyphen characters. |
|
api-version
|
query | True |
string minLength: 1 |
The API version to use for this operation. |
|
old
|
query |
string |
Existing test run identifier that should be rerun, if this is provided, the test will run with the JMX file, configuration and app components from the existing test run. You can override the configuration values for new test run in the request body. |
Request Body
Media Types: "application/merge-patch+json"
| Name | Type | Description |
|---|---|---|
| certificate |
Certificates metadata |
|
| description |
string maxLength: 100 |
The test run description. |
| displayName |
string minLength: 2maxLength: 50 |
Display name of a testRun. |
| environmentVariables |
object |
Environment variables which are defined as a set of <name,value> pairs. |
| passFailCriteria |
Pass fail criteria for a test. |
|
| secrets |
<string, Secret> |
Secrets can be stored in an Azure Key Vault or any other secret store. If the secret is stored in an Azure Key Vault, the value should be the secret identifier and the type should be AKV_SECRET_URI. If the secret is stored elsewhere, the secret value should be provided directly and the type should be SECRET_VALUE. |
| testId |
string minLength: 2maxLength: 50 |
Associated test Id. |
Responses
| Name | Type | Description |
|---|---|---|
| 200 OK |
The request has succeeded. |
|
| 201 Created |
The request has succeeded and a new resource has been created as a result. |
|
| Other Status Codes |
An unexpected error response. Headers x-ms-error-code: string |
Security
Oauth2
OAuth 2.0 Flow with Microsoft Entra ID.
Type:
oauth2
Flow:
implicit
Authorization URL:
https://login.microsoftonline.com/common/oauth2/v2.0/authorize
Scopes
| Name | Description |
|---|---|
| https://cnt-prod.loadtesting.azure.com/.default |
Examples
Create/update and start a load test run
Sample request
PATCH https://{endpoint}/test-runs/12316678-1234-1234-1234-122451189012?api-version=2022-11-01&oldTestRunId=12345678-1234-1234-1234-123456789012
{
"displayName": "Performance_LoadTest_Run1",
"testId": "12345678-1234-1234-1234-123456789012",
"description": "sample description",
"passFailCriteria": {
"passFailMetrics": {
"fefd759d-7fe8-4f83-8b6d-aeebe0f491fe": {
"clientMetric": "response_time_ms",
"aggregate": "percentage",
"condition": ">",
"value": 20,
"action": "continue"
}
}
},
"secrets": {
"secret1": {
"value": "https://samplevault.vault.azure.net/secrets/samplesecret/f113f91fd4c44a368049849c164db827",
"type": "AKV_SECRET_URI"
}
},
"environmentVariables": {
"envvar1": "sampletext"
}
}
Sample response
{
"testRunId": "12316678-1234-1234-1234-122451189012",
"displayName": "Performance_LoadTest_Run1",
"testId": "12345678-1234-1234-1234-123456789012",
"description": "sample description",
"status": "ACCEPTED",
"startDateTime": "2021-12-05T16:43:48.125Z",
"endDateTime": "2021-12-05T16:43:48.125Z",
"loadTestConfiguration": {
"engineInstances": 6,
"splitAllCSVs": true
},
"testResult": "PASSED",
"passFailCriteria": {
"passFailMetrics": {
"fefd759d-7fe8-4f83-8b6d-aeebe0f491fe": {
"clientMetric": "response_time_ms",
"aggregate": "percentage",
"condition": ">",
"value": 20,
"action": "continue",
"actualValue": 10,
"result": "passed"
}
}
},
"testArtifacts": {
"inputArtifacts": {
"configFileInfo": {
"url": "https://dummyurl.com/configresource",
"fileName": "config.yaml",
"fileType": "ADDITIONAL_ARTIFACTS",
"expireDateTime": "2021-12-05T16:43:46.072Z",
"validationStatus": ""
},
"testScriptFileInfo": {
"url": "https://dummyurl.com/testscriptresource",
"fileName": "sample.jmx",
"fileType": "JMX_FILE",
"expireDateTime": "2021-12-05T16:43:46.072Z",
"validationStatus": "VALIDATION_SUCCESS"
},
"userPropFileInfo": {
"url": "https://dummyurl.com/userpropresource",
"fileName": "user.properties",
"fileType": "USER_PROPERTIES",
"expireDateTime": "2021-12-05T16:43:46.072Z",
"validationStatus": ""
},
"inputArtifactsZipFileInfo": {
"url": "https://dummyurl.com/inputartifactzipresource",
"fileName": "inputartifacts.zip",
"fileType": "ADDITIONAL_ARTIFACTS",
"expireDateTime": "2021-12-05T16:43:46.072Z",
"validationStatus": ""
},
"additionalFileInfo": []
},
"outputArtifacts": {
"resultFileInfo": {
"url": "https://dummyurl.com/dummyresourceresult",
"fileName": "results.zip",
"fileType": "ADDITIONAL_ARTIFACTS",
"expireDateTime": "2021-12-05T16:43:46.072Z",
"validationStatus": "VALIDATION_SUCCESS"
},
"logsFileInfo": {
"url": "https://dummyurl.com/logresource",
"fileName": "worker.log",
"fileType": "ADDITIONAL_ARTIFACTS",
"expireDateTime": "2022-05-16T09:14:46.0411793+00:00",
"validationStatus": ""
}
}
},
"executedDateTime": "2021-12-05T16:43:48.125Z",
"virtualUsers": 4,
"testRunStatistics": {
"Total": {
"transaction": "Total",
"sampleCount": 18,
"errorCount": 19,
"errorPct": 17,
"meanResTime": 13,
"medianResTime": 10,
"maxResTime": 16,
"minResTime": 18,
"pct1ResTime": 27,
"pct2ResTime": 20,
"pct3ResTime": 3,
"throughput": 5,
"receivedKBytesPerSec": 13,
"sentKBytesPerSec": 4
}
},
"createdDateTime": "2021-12-05T16:43:46.072Z",
"createdBy": "user@contoso.com",
"lastModifiedDateTime": "2021-12-05T16:43:46.072Z",
"lastModifiedBy": "user@contoso.com",
"portalUrl": "https://portal.azure.com/dummyresource",
"secrets": {
"secret1": {
"value": "https://samplevault.vault.azure.net/secrets/samplesecret/f113f91fd4c44a368049849c164db827",
"type": "AKV_SECRET_URI"
}
},
"environmentVariables": {
"envvar1": "sampletext"
},
"duration": 18,
"subnetId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/samplerg/providers/Microsoft.Network/virtualNetworks/samplenetworkresource/subnets/AAAAA0A0A0"
}
{
"testRunId": "12316678-1234-1234-1234-122451189012",
"displayName": "Performance_LoadTest_Run1",
"testId": "12345678-1234-1234-1234-123456789012",
"description": "sample description",
"status": "ACCEPTED",
"startDateTime": "2021-12-05T16:43:48.125Z",
"endDateTime": "2021-12-05T16:43:48.125Z",
"loadTestConfiguration": {
"engineInstances": 6,
"splitAllCSVs": true
},
"testResult": "PASSED",
"passFailCriteria": {
"passFailMetrics": {
"fefd759d-7fe8-4f83-8b6d-aeebe0f491fe": {
"clientMetric": "response_time_ms",
"aggregate": "percentage",
"condition": ">",
"value": 20,
"action": "continue",
"actualValue": 10,
"result": "passed"
}
}
},
"testArtifacts": {
"inputArtifacts": {
"configFileInfo": {
"url": "https://dummyurl.com/configresource",
"fileName": "config.yaml",
"fileType": "ADDITIONAL_ARTIFACTS",
"expireDateTime": "2021-12-05T16:43:46.072Z",
"validationStatus": ""
},
"testScriptFileInfo": {
"url": "https://dummyurl.com/testscriptresource",
"fileName": "sample.jmx",
"fileType": "JMX_FILE",
"expireDateTime": "2021-12-05T16:43:46.072Z",
"validationStatus": "VALIDATION_SUCCESS"
},
"userPropFileInfo": {
"url": "https://dummyurl.com/userpropresource",
"fileName": "user.properties",
"fileType": "USER_PROPERTIES",
"expireDateTime": "2021-12-05T16:43:46.072Z",
"validationStatus": ""
},
"inputArtifactsZipFileInfo": {
"url": "https://dummyurl.com/inputartifactzipresource",
"fileName": "inputartifacts.zip",
"fileType": "ADDITIONAL_ARTIFACTS",
"expireDateTime": "2021-12-05T16:43:46.072Z",
"validationStatus": ""
},
"additionalFileInfo": []
},
"outputArtifacts": {
"resultFileInfo": {
"url": "https://dummyurl.com/dummyresourceresult",
"fileName": "results.zip",
"fileType": "ADDITIONAL_ARTIFACTS",
"expireDateTime": "2021-12-05T16:43:46.072Z",
"validationStatus": "VALIDATION_SUCCESS"
},
"logsFileInfo": {
"url": "https://dummyurl.com/logresource",
"fileName": "worker.log",
"fileType": "ADDITIONAL_ARTIFACTS",
"expireDateTime": "2022-05-16T09:14:46.0411793+00:00",
"validationStatus": ""
}
}
},
"executedDateTime": "2021-12-05T16:43:48.125Z",
"virtualUsers": 4,
"testRunStatistics": {
"Total": {
"transaction": "Total",
"sampleCount": 18,
"errorCount": 19,
"errorPct": 17,
"meanResTime": 13,
"medianResTime": 10,
"maxResTime": 16,
"minResTime": 18,
"pct1ResTime": 27,
"pct2ResTime": 20,
"pct3ResTime": 3,
"throughput": 5,
"receivedKBytesPerSec": 13,
"sentKBytesPerSec": 4
}
},
"createdDateTime": "2021-12-05T16:43:46.072Z",
"createdBy": "user@contoso.com",
"lastModifiedDateTime": "2021-12-05T16:43:46.072Z",
"lastModifiedBy": "user@contoso.com",
"portalUrl": "https://portal.azure.com/dummyresource",
"secrets": {
"secret1": {
"value": "https://samplevault.vault.azure.net/secrets/samplesecret/f113f91fd4c44a368049849c164db827",
"type": "AKV_SECRET_URI"
}
},
"environmentVariables": {
"envvar1": "sampletext"
},
"duration": 18,
"subnetId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/samplerg/providers/Microsoft.Network/virtualNetworks/samplenetworkresource/subnets/AAAAA0A0A0"
}
Definitions
| Name | Description |
|---|---|
|
Azure. |
The error object. |
|
Azure. |
A response containing error details. |
|
Azure. |
An object containing more specific information about the error. As per Azure REST API guidelines - https://aka.ms/AzureRestApiGuidelines#handling-errors. |
|
Certificate |
Certificates metadata |
|
Certificate |
Types of certificates supported. |
|
Error |
Error details if there is any failure in load test run |
|
File |
Types of file supported. |
|
File |
File status. |
|
Load |
Configurations for the load test. |
|
Optional |
Configuration for quick load test |
|
Pass |
Action taken after the threshold is met. Default is ‘continue’. |
|
Pass |
Aggregation functions for pass/fail criteria. |
|
Pass |
Pass fail criteria for a test. |
|
Pass |
Pass fail metric |
|
Pass |
Pass/fail criteria result. |
|
Pass |
Test result based on pass/fail criteria. |
| PFMetrics |
Metrics for pass/fail criteria. |
| Secret |
Secret |
|
Secret |
Types of secrets supported. |
|
Test |
Load test run model |
|
Test |
Collection of test run artifacts |
|
Test |
Load test run model |
|
Test |
Test run file info. |
|
Test |
The input artifacts for the test run. |
|
Test |
The output artifacts for the test run. |
|
Test |
Test run statistics. |
|
Test |
Test run status. |
Azure.Core.Foundations.Error
The error object.
| Name | Type | Description |
|---|---|---|
| code |
string |
One of a server-defined set of error codes. |
| details |
An array of details about specific errors that led to this reported error. |
|
| innererror |
An object containing more specific information than the current object about the error. |
|
| message |
string |
A human-readable representation of the error. |
| target |
string |
The target of the error. |
Azure.Core.Foundations.ErrorResponse
A response containing error details.
| Name | Type | Description |
|---|---|---|
| error |
The error object. |
Azure.Core.Foundations.InnerError
An object containing more specific information about the error. As per Azure REST API guidelines - https://aka.ms/AzureRestApiGuidelines#handling-errors.
| Name | Type | Description |
|---|---|---|
| code |
string |
One of a server-defined set of error codes. |
| innererror |
Inner error. |
CertificateMetadata
Certificates metadata
| Name | Type | Description |
|---|---|---|
| name |
string |
Name of the certificate. |
| type |
Type of certificate |
|
| value |
string |
The value of the certificate for respective type |
CertificateType
Types of certificates supported.
| Value | Description |
|---|---|
| AKV_CERT_URI |
If the certificate is stored in an Azure Key Vault. |
ErrorDetails
Error details if there is any failure in load test run
| Name | Type | Description |
|---|---|---|
| message |
string |
Error details in case test run was not successfully run. |
FileType
Types of file supported.
| Value | Description |
|---|---|
| JMX_FILE |
If the file is a JMX script. |
| USER_PROPERTIES |
If the file is a user properties file. |
| ADDITIONAL_ARTIFACTS |
If the file is not among any of the other supported file types. |
FileValidationStatus
File status.
| Value | Description |
|---|---|
| NOT_VALIDATED |
File is not validated. |
| VALIDATION_SUCCESS |
File is validated. |
| VALIDATION_FAILURE |
File validation is failed. |
| VALIDATION_INITIATED |
File validation is in progress. |
| VALIDATION_NOT_REQUIRED |
Validation is not required. |
LoadTestConfiguration
Configurations for the load test.
| Name | Type | Default value | Description |
|---|---|---|---|
| engineInstances |
integer (int32) |
The number of engine instances to execute load test. Supported values are in range of 1-400. Required for creating a new test. |
|
| optionalLoadTestConfig |
Configuration for quick load test |
||
| quickStartTest |
boolean |
False |
If true, optionalLoadTestConfig is required and JMX script for the load test is not required to upload. |
| splitAllCSVs |
boolean |
False |
If false, Azure Load Testing copies and processes your input files unmodified across all test engine instances. If true, Azure Load Testing splits the CSV input data evenly across all engine instances. If you provide multiple CSV files, each file will be split evenly. |
OptionalLoadTestConfiguration
Configuration for quick load test
| Name | Type | Description |
|---|---|---|
| duration |
integer (int32) |
Test run duration in seconds. |
| endpointUrl |
string |
Test URL. Provide the complete HTTP URL. For example, |
| rampUpTime |
integer (int32) |
Ramp up time in seconds. |
| virtualUsers |
integer (int32) |
No of concurrent virtual users. |
PassFailAction
Action taken after the threshold is met. Default is ‘continue’.
| Value | Description |
|---|---|
| continue |
Test will continue to run even if pass fail metric criteria metric gets failed. |
| stop |
Test run will stop if pass fail criteria metric is not passed. |
PassFailAggregationFunction
Aggregation functions for pass/fail criteria.
| Value | Description |
|---|---|
| count |
Criteria applies for count value. |
| percentage |
Criteria applies for given percentage value. |
| avg |
Criteria applies for avg value. |
| p50 |
Criteria applies for 50th percentile value. |
| p90 |
Criteria applies for 90th percentile value. |
| p95 |
Criteria applies for 95th percentile value. |
| p99 |
Criteria applies for 99th percentile value. |
| min |
Criteria applies for minimum value. |
| max |
Criteria applies for maximum value. |
PassFailCriteria
Pass fail criteria for a test.
| Name | Type | Description |
|---|---|---|
| passFailMetrics |
<string,
Pass |
Map of id and pass fail metrics { id : pass fail metrics }. |
PassFailMetric
Pass fail metric
| Name | Type | Default value | Description |
|---|---|---|---|
| action | continue |
Action taken after the threshold is met. Default is ‘continue’. |
|
| actualValue |
number (double) |
The actual value of the client metric for the test run. |
|
| aggregate |
The aggregation function to be applied on the client metric. Allowed functions
|
||
| clientMetric |
The client metric on which the criteria should be applied. |
||
| condition |
string |
The comparison operator. Supported types ‘>’, ‘<’ |
|
| requestName |
string |
Request name for which the Pass fail criteria has to be applied |
|
| result |
Outcome of the test run. |
||
| value |
number (double) |
The value to compare with the client metric. Allowed values - ‘error : [0.0 , 100.0] unit- % ’, response_time_ms and latency : any integer value unit- ms. |
PassFailResult
Pass/fail criteria result.
| Value | Description |
|---|---|
| passed |
Given pass fail criteria metric has passed. |
| undetermined |
Given pass fail criteria metric couldn't determine. |
| failed |
Given pass fail criteria metric has failed. |
PassFailTestResult
Test result based on pass/fail criteria.
| Value | Description |
|---|---|
| PASSED |
Pass/fail criteria has passed. |
| NOT_APPLICABLE |
Pass/fail criteria is not applicable. |
| FAILED |
Pass/fail criteria has failed. |
PFMetrics
Metrics for pass/fail criteria.
| Value | Description |
|---|---|
| response_time_ms |
Pass fail criteria for response time metric in milliseconds. |
| latency |
Pass fail criteria for latency metric in milliseconds. |
| error |
Pass fail criteria for error metric. |
| requests |
Pass fail criteria for total requests. |
| requests_per_sec |
Pass fail criteria for request per second. |
Secret
Secret
| Name | Type | Description |
|---|---|---|
| type |
Type of secret |
|
| value |
string |
The value of the secret for the respective type |
SecretType
Types of secrets supported.
| Value | Description |
|---|---|
| AKV_SECRET_URI |
If the secret is stored in an Azure Key Vault. |
| SECRET_VALUE |
If the secret value provided as plain text. |
TestRun
Load test run model
| Name | Type | Description |
|---|---|---|
| certificate |
Certificates metadata |
|
| createdBy |
string |
The user that created. |
| createdDateTime |
string (date-time) |
The creation datetime(RFC 3339 literal format). |
| description |
string maxLength: 100 |
The test run description. |
| displayName |
string minLength: 2maxLength: 50 |
Display name of a testRun. |
| duration |
integer (int64) |
Test run duration in milliseconds. |
| endDateTime |
string (date-time) |
The test run end DateTime(RFC 3339 literal format). |
| environmentVariables |
object |
Environment variables which are defined as a set of <name,value> pairs. |
| errorDetails |
Error details if there is any failure in load test run |
|
| executedDateTime |
string (date-time) |
Test run initiated time. |
| lastModifiedBy |
string |
The user that last modified. |
| lastModifiedDateTime |
string (date-time) |
The last Modified datetime(RFC 3339 literal format). |
| loadTestConfiguration |
The load test configuration. |
|
| passFailCriteria |
Pass fail criteria for a test. |
|
| portalUrl |
string |
Portal url. |
| secrets |
<string, Secret> |
Secrets can be stored in an Azure Key Vault or any other secret store. If the secret is stored in an Azure Key Vault, the value should be the secret identifier and the type should be AKV_SECRET_URI. If the secret is stored elsewhere, the secret value should be provided directly and the type should be SECRET_VALUE. |
| startDateTime |
string (date-time) |
The test run start DateTime(RFC 3339 literal format). |
| status |
The test run status. |
|
| subnetId |
string |
Subnet ID on which the load test instances should run. |
| testArtifacts |
Collection of test run artifacts |
|
| testId |
string minLength: 2maxLength: 50 |
Associated test Id. |
| testResult |
Test result for pass/Fail criteria used during the test run. |
|
| testRunId |
string minLength: 2maxLength: 50 pattern: ^[a-z0-9_-]*$ |
Unique test run identifier for the load test run, must contain only lower-case alphabetic, numeric, underscore or hyphen characters. |
| testRunStatistics |
<string,
Test |
Test run statistics. Key is the sampler name and value is the set of statistics for performance metrics like response time, throughput, etc. from the load test run. The sampler name is the same as the name mentioned in the test script. Sampler name "Total" represents the aggregated statistics of all the samplers. |
| virtualUsers |
integer (int32) |
Number of virtual users, for which test has been run. |
TestRunArtifacts
Collection of test run artifacts
| Name | Type | Description |
|---|---|---|
| inputArtifacts |
The input artifacts for the test run. |
|
| outputArtifacts |
The output artifacts for the test run. |
TestRunCreateOrUpdate
Load test run model
| Name | Type | Description |
|---|---|---|
| certificate |
Certificates metadata |
|
| description |
string maxLength: 100 |
The test run description. |
| displayName |
string minLength: 2maxLength: 50 |
Display name of a testRun. |
| environmentVariables |
object |
Environment variables which are defined as a set of <name,value> pairs. |
| passFailCriteria |
Pass fail criteria for a test. |
|
| secrets |
<string, Secret> |
Secrets can be stored in an Azure Key Vault or any other secret store. If the secret is stored in an Azure Key Vault, the value should be the secret identifier and the type should be AKV_SECRET_URI. If the secret is stored elsewhere, the secret value should be provided directly and the type should be SECRET_VALUE. |
| testId |
string minLength: 2maxLength: 50 |
Associated test Id. |
TestRunFileInfo
Test run file info.
| Name | Type | Description |
|---|---|---|
| expireDateTime |
string (date-time) |
Expiry time of the file (RFC 3339 literal format) |
| fileName |
string |
Name of the file. |
| fileType |
File type |
|
| url |
string (uri) |
File URL. |
| validationFailureDetails |
string |
Validation failure error details |
| validationStatus |
Validation status of the file |
TestRunInputArtifacts
The input artifacts for the test run.
| Name | Type | Description |
|---|---|---|
| additionalFileInfo |
Additional supported files for the test run |
|
| configFileInfo |
The load test YAML file that contains the the test configuration |
|
| inputArtifactsZipFileInfo |
The zip file for all input artifacts |
|
| testScriptFileInfo |
The test script file for the test run |
|
| userPropFileInfo |
The user properties file |
TestRunOutputArtifacts
The output artifacts for the test run.
| Name | Type | Description |
|---|---|---|
| logsFileInfo |
The test run report with metrics |
|
| resultFileInfo |
The test run results file |
TestRunStatistics
Test run statistics.
| Name | Type | Description |
|---|---|---|
| errorCount |
number (double) |
Error count. |
| errorPct |
number (double) |
Error percentage. |
| maxResTime |
number (double) |
Max response time. |
| meanResTime |
number (double) |
Mean response time. |
| medianResTime |
number (double) |
Median response time. |
| minResTime |
number (double) |
Minimum response time. |
| pct1ResTime |
number (double) |
90 percentile response time. |
| pct2ResTime |
number (double) |
95 percentile response time. |
| pct3ResTime |
number (double) |
99 percentile response time. |
| receivedKBytesPerSec |
number (double) |
Received network bytes. |
| sampleCount |
number (double) |
Sampler count. |
| sentKBytesPerSec |
number (double) |
Send network bytes. |
| throughput |
number (double) |
Throughput. |
| transaction |
string |
Transaction name. |
TestRunStatus
Test run status.
| Value | Description |
|---|---|
| ACCEPTED |
Test run request is accepted. |
| NOTSTARTED |
Test run is not yet started. |
| PROVISIONING |
Test run is provisioning. |
| PROVISIONED |
Test run is provisioned. |
| CONFIGURING |
Test run is getting configured. |
| CONFIGURED |
Test run configuration is done. |
| EXECUTING |
Test run has started executing. |
| EXECUTED |
Test run execution is completed. |
| DEPROVISIONING |
Test run is getting deprovisioned. |
| DEPROVISIONED |
Test run is deprovisioned. |
| DONE |
Test run is completed. |
| CANCELLING |
Test run is being cancelled. |
| CANCELLED |
Test run request is cancelled. |
| FAILED |
Test run request is failed. |
| VALIDATION_SUCCESS |
Test run JMX file is validated. |
| VALIDATION_FAILURE |
Test run JMX file validation is failed. |