Share via


Load Test Run - Create Or Update Test Run

Create and start a new test run with the given test run Id.

PATCH https://{endpoint}/test-runs/{testRunId}?api-version=2022-11-01
PATCH https://{endpoint}/test-runs/{testRunId}?api-version=2022-11-01&oldTestRunId={oldTestRunId}

URI Parameters

Name In Required Type Description
endpoint
path True

string

testRunId
path True

string

minLength: 2
maxLength: 50
pattern: ^[a-z0-9_-]*$

Unique test run identifier for the load test run, must contain only lower-case alphabetic, numeric, underscore or hyphen characters.

api-version
query True

string

minLength: 1

The API version to use for this operation.

oldTestRunId
query

string

Existing test run identifier that should be rerun, if this is provided, the test will run with the JMX file, configuration and app components from the existing test run. You can override the configuration values for new test run in the request body.

Request Body

Media Types: "application/merge-patch+json"

Name Type Description
certificate

CertificateMetadata

Certificates metadata

description

string

maxLength: 100

The test run description.

displayName

string

minLength: 2
maxLength: 50

Display name of a testRun.

environmentVariables

object

Environment variables which are defined as a set of <name,value> pairs.

passFailCriteria

PassFailCriteria

Pass fail criteria for a test.

secrets

<string,  Secret>

Secrets can be stored in an Azure Key Vault or any other secret store. If the secret is stored in an Azure Key Vault, the value should be the secret identifier and the type should be AKV_SECRET_URI. If the secret is stored elsewhere, the secret value should be provided directly and the type should be SECRET_VALUE.

testId

string

minLength: 2
maxLength: 50

Associated test Id.

Responses

Name Type Description
200 OK

TestRun

The request has succeeded.

201 Created

TestRun

The request has succeeded and a new resource has been created as a result.

Other Status Codes

Azure.Core.Foundations.ErrorResponse

An unexpected error response.

Headers

x-ms-error-code: string

Security

Oauth2

OAuth 2.0 Flow with Microsoft Entra ID.

Type: oauth2
Flow: implicit
Authorization URL: https://login.microsoftonline.com/common/oauth2/v2.0/authorize

Scopes

Name Description
https://cnt-prod.loadtesting.azure.com/.default

Examples

Create/update and start a load test run

Sample request

PATCH https://{endpoint}/test-runs/12316678-1234-1234-1234-122451189012?api-version=2022-11-01&oldTestRunId=12345678-1234-1234-1234-123456789012

{
  "displayName": "Performance_LoadTest_Run1",
  "testId": "12345678-1234-1234-1234-123456789012",
  "description": "sample description",
  "passFailCriteria": {
    "passFailMetrics": {
      "fefd759d-7fe8-4f83-8b6d-aeebe0f491fe": {
        "clientMetric": "response_time_ms",
        "aggregate": "percentage",
        "condition": ">",
        "value": 20,
        "action": "continue"
      }
    }
  },
  "secrets": {
    "secret1": {
      "value": "https://samplevault.vault.azure.net/secrets/samplesecret/f113f91fd4c44a368049849c164db827",
      "type": "AKV_SECRET_URI"
    }
  },
  "environmentVariables": {
    "envvar1": "sampletext"
  }
}

Sample response

{
  "testRunId": "12316678-1234-1234-1234-122451189012",
  "displayName": "Performance_LoadTest_Run1",
  "testId": "12345678-1234-1234-1234-123456789012",
  "description": "sample description",
  "status": "ACCEPTED",
  "startDateTime": "2021-12-05T16:43:48.125Z",
  "endDateTime": "2021-12-05T16:43:48.125Z",
  "loadTestConfiguration": {
    "engineInstances": 6,
    "splitAllCSVs": true
  },
  "testResult": "PASSED",
  "passFailCriteria": {
    "passFailMetrics": {
      "fefd759d-7fe8-4f83-8b6d-aeebe0f491fe": {
        "clientMetric": "response_time_ms",
        "aggregate": "percentage",
        "condition": ">",
        "value": 20,
        "action": "continue",
        "actualValue": 10,
        "result": "passed"
      }
    }
  },
  "testArtifacts": {
    "inputArtifacts": {
      "configFileInfo": {
        "url": "https://dummyurl.com/configresource",
        "fileName": "config.yaml",
        "fileType": "ADDITIONAL_ARTIFACTS",
        "expireDateTime": "2021-12-05T16:43:46.072Z",
        "validationStatus": ""
      },
      "testScriptFileInfo": {
        "url": "https://dummyurl.com/testscriptresource",
        "fileName": "sample.jmx",
        "fileType": "JMX_FILE",
        "expireDateTime": "2021-12-05T16:43:46.072Z",
        "validationStatus": "VALIDATION_SUCCESS"
      },
      "userPropFileInfo": {
        "url": "https://dummyurl.com/userpropresource",
        "fileName": "user.properties",
        "fileType": "USER_PROPERTIES",
        "expireDateTime": "2021-12-05T16:43:46.072Z",
        "validationStatus": ""
      },
      "inputArtifactsZipFileInfo": {
        "url": "https://dummyurl.com/inputartifactzipresource",
        "fileName": "inputartifacts.zip",
        "fileType": "ADDITIONAL_ARTIFACTS",
        "expireDateTime": "2021-12-05T16:43:46.072Z",
        "validationStatus": ""
      },
      "additionalFileInfo": []
    },
    "outputArtifacts": {
      "resultFileInfo": {
        "url": "https://dummyurl.com/dummyresourceresult",
        "fileName": "results.zip",
        "fileType": "ADDITIONAL_ARTIFACTS",
        "expireDateTime": "2021-12-05T16:43:46.072Z",
        "validationStatus": "VALIDATION_SUCCESS"
      },
      "logsFileInfo": {
        "url": "https://dummyurl.com/logresource",
        "fileName": "worker.log",
        "fileType": "ADDITIONAL_ARTIFACTS",
        "expireDateTime": "2022-05-16T09:14:46.0411793+00:00",
        "validationStatus": ""
      }
    }
  },
  "executedDateTime": "2021-12-05T16:43:48.125Z",
  "virtualUsers": 4,
  "testRunStatistics": {
    "Total": {
      "transaction": "Total",
      "sampleCount": 18,
      "errorCount": 19,
      "errorPct": 17,
      "meanResTime": 13,
      "medianResTime": 10,
      "maxResTime": 16,
      "minResTime": 18,
      "pct1ResTime": 27,
      "pct2ResTime": 20,
      "pct3ResTime": 3,
      "throughput": 5,
      "receivedKBytesPerSec": 13,
      "sentKBytesPerSec": 4
    }
  },
  "createdDateTime": "2021-12-05T16:43:46.072Z",
  "createdBy": "user@contoso.com",
  "lastModifiedDateTime": "2021-12-05T16:43:46.072Z",
  "lastModifiedBy": "user@contoso.com",
  "portalUrl": "https://portal.azure.com/dummyresource",
  "secrets": {
    "secret1": {
      "value": "https://samplevault.vault.azure.net/secrets/samplesecret/f113f91fd4c44a368049849c164db827",
      "type": "AKV_SECRET_URI"
    }
  },
  "environmentVariables": {
    "envvar1": "sampletext"
  },
  "duration": 18,
  "subnetId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/samplerg/providers/Microsoft.Network/virtualNetworks/samplenetworkresource/subnets/AAAAA0A0A0"
}
{
  "testRunId": "12316678-1234-1234-1234-122451189012",
  "displayName": "Performance_LoadTest_Run1",
  "testId": "12345678-1234-1234-1234-123456789012",
  "description": "sample description",
  "status": "ACCEPTED",
  "startDateTime": "2021-12-05T16:43:48.125Z",
  "endDateTime": "2021-12-05T16:43:48.125Z",
  "loadTestConfiguration": {
    "engineInstances": 6,
    "splitAllCSVs": true
  },
  "testResult": "PASSED",
  "passFailCriteria": {
    "passFailMetrics": {
      "fefd759d-7fe8-4f83-8b6d-aeebe0f491fe": {
        "clientMetric": "response_time_ms",
        "aggregate": "percentage",
        "condition": ">",
        "value": 20,
        "action": "continue",
        "actualValue": 10,
        "result": "passed"
      }
    }
  },
  "testArtifacts": {
    "inputArtifacts": {
      "configFileInfo": {
        "url": "https://dummyurl.com/configresource",
        "fileName": "config.yaml",
        "fileType": "ADDITIONAL_ARTIFACTS",
        "expireDateTime": "2021-12-05T16:43:46.072Z",
        "validationStatus": ""
      },
      "testScriptFileInfo": {
        "url": "https://dummyurl.com/testscriptresource",
        "fileName": "sample.jmx",
        "fileType": "JMX_FILE",
        "expireDateTime": "2021-12-05T16:43:46.072Z",
        "validationStatus": "VALIDATION_SUCCESS"
      },
      "userPropFileInfo": {
        "url": "https://dummyurl.com/userpropresource",
        "fileName": "user.properties",
        "fileType": "USER_PROPERTIES",
        "expireDateTime": "2021-12-05T16:43:46.072Z",
        "validationStatus": ""
      },
      "inputArtifactsZipFileInfo": {
        "url": "https://dummyurl.com/inputartifactzipresource",
        "fileName": "inputartifacts.zip",
        "fileType": "ADDITIONAL_ARTIFACTS",
        "expireDateTime": "2021-12-05T16:43:46.072Z",
        "validationStatus": ""
      },
      "additionalFileInfo": []
    },
    "outputArtifacts": {
      "resultFileInfo": {
        "url": "https://dummyurl.com/dummyresourceresult",
        "fileName": "results.zip",
        "fileType": "ADDITIONAL_ARTIFACTS",
        "expireDateTime": "2021-12-05T16:43:46.072Z",
        "validationStatus": "VALIDATION_SUCCESS"
      },
      "logsFileInfo": {
        "url": "https://dummyurl.com/logresource",
        "fileName": "worker.log",
        "fileType": "ADDITIONAL_ARTIFACTS",
        "expireDateTime": "2022-05-16T09:14:46.0411793+00:00",
        "validationStatus": ""
      }
    }
  },
  "executedDateTime": "2021-12-05T16:43:48.125Z",
  "virtualUsers": 4,
  "testRunStatistics": {
    "Total": {
      "transaction": "Total",
      "sampleCount": 18,
      "errorCount": 19,
      "errorPct": 17,
      "meanResTime": 13,
      "medianResTime": 10,
      "maxResTime": 16,
      "minResTime": 18,
      "pct1ResTime": 27,
      "pct2ResTime": 20,
      "pct3ResTime": 3,
      "throughput": 5,
      "receivedKBytesPerSec": 13,
      "sentKBytesPerSec": 4
    }
  },
  "createdDateTime": "2021-12-05T16:43:46.072Z",
  "createdBy": "user@contoso.com",
  "lastModifiedDateTime": "2021-12-05T16:43:46.072Z",
  "lastModifiedBy": "user@contoso.com",
  "portalUrl": "https://portal.azure.com/dummyresource",
  "secrets": {
    "secret1": {
      "value": "https://samplevault.vault.azure.net/secrets/samplesecret/f113f91fd4c44a368049849c164db827",
      "type": "AKV_SECRET_URI"
    }
  },
  "environmentVariables": {
    "envvar1": "sampletext"
  },
  "duration": 18,
  "subnetId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/samplerg/providers/Microsoft.Network/virtualNetworks/samplenetworkresource/subnets/AAAAA0A0A0"
}

Definitions

Name Description
Azure.Core.Foundations.Error

The error object.

Azure.Core.Foundations.ErrorResponse

A response containing error details.

Azure.Core.Foundations.InnerError

An object containing more specific information about the error. As per Azure REST API guidelines - https://aka.ms/AzureRestApiGuidelines#handling-errors.

CertificateMetadata

Certificates metadata

CertificateType

Types of certificates supported.

ErrorDetails

Error details if there is any failure in load test run

FileType

Types of file supported.

FileValidationStatus

File status.

LoadTestConfiguration

Configurations for the load test.

OptionalLoadTestConfiguration

Configuration for quick load test

PassFailAction

Action taken after the threshold is met. Default is ‘continue’.

PassFailAggregationFunction

Aggregation functions for pass/fail criteria.

PassFailCriteria

Pass fail criteria for a test.

PassFailMetric

Pass fail metric

PassFailResult

Pass/fail criteria result.

PassFailTestResult

Test result based on pass/fail criteria.

PFMetrics

Metrics for pass/fail criteria.

Secret

Secret

SecretType

Types of secrets supported.

TestRun

Load test run model

TestRunArtifacts

Collection of test run artifacts

TestRunCreateOrUpdate

Load test run model

TestRunFileInfo

Test run file info.

TestRunInputArtifacts

The input artifacts for the test run.

TestRunOutputArtifacts

The output artifacts for the test run.

TestRunStatistics

Test run statistics.

TestRunStatus

Test run status.

Azure.Core.Foundations.Error

The error object.

Name Type Description
code

string

One of a server-defined set of error codes.

details

Azure.Core.Foundations.Error[]

An array of details about specific errors that led to this reported error.

innererror

Azure.Core.Foundations.InnerError

An object containing more specific information than the current object about the error.

message

string

A human-readable representation of the error.

target

string

The target of the error.

Azure.Core.Foundations.ErrorResponse

A response containing error details.

Name Type Description
error

Azure.Core.Foundations.Error

The error object.

Azure.Core.Foundations.InnerError

An object containing more specific information about the error. As per Azure REST API guidelines - https://aka.ms/AzureRestApiGuidelines#handling-errors.

Name Type Description
code

string

One of a server-defined set of error codes.

innererror

Azure.Core.Foundations.InnerError

Inner error.

CertificateMetadata

Certificates metadata

Name Type Description
name

string

Name of the certificate.

type

CertificateType

Type of certificate

value

string

The value of the certificate for respective type

CertificateType

Types of certificates supported.

Value Description
AKV_CERT_URI

If the certificate is stored in an Azure Key Vault.

ErrorDetails

Error details if there is any failure in load test run

Name Type Description
message

string

Error details in case test run was not successfully run.

FileType

Types of file supported.

Value Description
JMX_FILE

If the file is a JMX script.

USER_PROPERTIES

If the file is a user properties file.

ADDITIONAL_ARTIFACTS

If the file is not among any of the other supported file types.

FileValidationStatus

File status.

Value Description
NOT_VALIDATED

File is not validated.

VALIDATION_SUCCESS

File is validated.

VALIDATION_FAILURE

File validation is failed.

VALIDATION_INITIATED

File validation is in progress.

VALIDATION_NOT_REQUIRED

Validation is not required.

LoadTestConfiguration

Configurations for the load test.

Name Type Default value Description
engineInstances

integer (int32)

The number of engine instances to execute load test. Supported values are in range of 1-400. Required for creating a new test.

optionalLoadTestConfig

OptionalLoadTestConfiguration

Configuration for quick load test

quickStartTest

boolean

False

If true, optionalLoadTestConfig is required and JMX script for the load test is not required to upload.

splitAllCSVs

boolean

False

If false, Azure Load Testing copies and processes your input files unmodified across all test engine instances. If true, Azure Load Testing splits the CSV input data evenly across all engine instances. If you provide multiple CSV files, each file will be split evenly.

OptionalLoadTestConfiguration

Configuration for quick load test

Name Type Description
duration

integer (int32)

Test run duration in seconds.

endpointUrl

string

Test URL. Provide the complete HTTP URL. For example, https://contoso-app.azurewebsites.net/login

rampUpTime

integer (int32)

Ramp up time in seconds.

virtualUsers

integer (int32)

No of concurrent virtual users.

PassFailAction

Action taken after the threshold is met. Default is ‘continue’.

Value Description
continue

Test will continue to run even if pass fail metric criteria metric gets failed.

stop

Test run will stop if pass fail criteria metric is not passed.

PassFailAggregationFunction

Aggregation functions for pass/fail criteria.

Value Description
count

Criteria applies for count value.

percentage

Criteria applies for given percentage value.

avg

Criteria applies for avg value.

p50

Criteria applies for 50th percentile value.

p90

Criteria applies for 90th percentile value.

p95

Criteria applies for 95th percentile value.

p99

Criteria applies for 99th percentile value.

min

Criteria applies for minimum value.

max

Criteria applies for maximum value.

PassFailCriteria

Pass fail criteria for a test.

Name Type Description
passFailMetrics

<string,  PassFailMetric>

Map of id and pass fail metrics { id : pass fail metrics }.

PassFailMetric

Pass fail metric

Name Type Default value Description
action

PassFailAction

continue

Action taken after the threshold is met. Default is ‘continue’.

actualValue

number (double)

The actual value of the client metric for the test run.

aggregate

PassFailAggregationFunction

The aggregation function to be applied on the client metric. Allowed functions

  • ‘percentage’ - for error metric , ‘avg’, percentiles like ‘p50’, ‘p90’, & so on, ‘min’, ‘max’ - for response_time_ms and latency metric, ‘avg’ - for requests_per_sec, ‘count’ - for requests
clientMetric

PFMetrics

The client metric on which the criteria should be applied.

condition

string

The comparison operator. Supported types ‘>’, ‘<’

requestName

string

Request name for which the Pass fail criteria has to be applied

result

PassFailResult

Outcome of the test run.

value

number (double)

The value to compare with the client metric. Allowed values - ‘error : [0.0 , 100.0] unit- % ’, response_time_ms and latency : any integer value unit- ms.

PassFailResult

Pass/fail criteria result.

Value Description
passed

Given pass fail criteria metric has passed.

undetermined

Given pass fail criteria metric couldn't determine.

failed

Given pass fail criteria metric has failed.

PassFailTestResult

Test result based on pass/fail criteria.

Value Description
PASSED

Pass/fail criteria has passed.

NOT_APPLICABLE

Pass/fail criteria is not applicable.

FAILED

Pass/fail criteria has failed.

PFMetrics

Metrics for pass/fail criteria.

Value Description
response_time_ms

Pass fail criteria for response time metric in milliseconds.

latency

Pass fail criteria for latency metric in milliseconds.

error

Pass fail criteria for error metric.

requests

Pass fail criteria for total requests.

requests_per_sec

Pass fail criteria for request per second.

Secret

Secret

Name Type Description
type

SecretType

Type of secret

value

string

The value of the secret for the respective type

SecretType

Types of secrets supported.

Value Description
AKV_SECRET_URI

If the secret is stored in an Azure Key Vault.

SECRET_VALUE

If the secret value provided as plain text.

TestRun

Load test run model

Name Type Description
certificate

CertificateMetadata

Certificates metadata

createdBy

string

The user that created.

createdDateTime

string (date-time)

The creation datetime(RFC 3339 literal format).

description

string

maxLength: 100

The test run description.

displayName

string

minLength: 2
maxLength: 50

Display name of a testRun.

duration

integer (int64)

Test run duration in milliseconds.

endDateTime

string (date-time)

The test run end DateTime(RFC 3339 literal format).

environmentVariables

object

Environment variables which are defined as a set of <name,value> pairs.

errorDetails

ErrorDetails[]

Error details if there is any failure in load test run

executedDateTime

string (date-time)

Test run initiated time.

lastModifiedBy

string

The user that last modified.

lastModifiedDateTime

string (date-time)

The last Modified datetime(RFC 3339 literal format).

loadTestConfiguration

LoadTestConfiguration

The load test configuration.

passFailCriteria

PassFailCriteria

Pass fail criteria for a test.

portalUrl

string

Portal url.

secrets

<string,  Secret>

Secrets can be stored in an Azure Key Vault or any other secret store. If the secret is stored in an Azure Key Vault, the value should be the secret identifier and the type should be AKV_SECRET_URI. If the secret is stored elsewhere, the secret value should be provided directly and the type should be SECRET_VALUE.

startDateTime

string (date-time)

The test run start DateTime(RFC 3339 literal format).

status

TestRunStatus

The test run status.

subnetId

string

Subnet ID on which the load test instances should run.

testArtifacts

TestRunArtifacts

Collection of test run artifacts

testId

string

minLength: 2
maxLength: 50

Associated test Id.

testResult

PassFailTestResult

Test result for pass/Fail criteria used during the test run.

testRunId

string

minLength: 2
maxLength: 50
pattern: ^[a-z0-9_-]*$

Unique test run identifier for the load test run, must contain only lower-case alphabetic, numeric, underscore or hyphen characters.

testRunStatistics

<string,  TestRunStatistics>

Test run statistics. Key is the sampler name and value is the set of statistics for performance metrics like response time, throughput, etc. from the load test run. The sampler name is the same as the name mentioned in the test script. Sampler name "Total" represents the aggregated statistics of all the samplers.

virtualUsers

integer (int32)

Number of virtual users, for which test has been run.

TestRunArtifacts

Collection of test run artifacts

Name Type Description
inputArtifacts

TestRunInputArtifacts

The input artifacts for the test run.

outputArtifacts

TestRunOutputArtifacts

The output artifacts for the test run.

TestRunCreateOrUpdate

Load test run model

Name Type Description
certificate

CertificateMetadata

Certificates metadata

description

string

maxLength: 100

The test run description.

displayName

string

minLength: 2
maxLength: 50

Display name of a testRun.

environmentVariables

object

Environment variables which are defined as a set of <name,value> pairs.

passFailCriteria

PassFailCriteria

Pass fail criteria for a test.

secrets

<string,  Secret>

Secrets can be stored in an Azure Key Vault or any other secret store. If the secret is stored in an Azure Key Vault, the value should be the secret identifier and the type should be AKV_SECRET_URI. If the secret is stored elsewhere, the secret value should be provided directly and the type should be SECRET_VALUE.

testId

string

minLength: 2
maxLength: 50

Associated test Id.

TestRunFileInfo

Test run file info.

Name Type Description
expireDateTime

string (date-time)

Expiry time of the file (RFC 3339 literal format)

fileName

string

Name of the file.

fileType

FileType

File type

url

string (uri)

File URL.

validationFailureDetails

string

Validation failure error details

validationStatus

FileValidationStatus

Validation status of the file

TestRunInputArtifacts

The input artifacts for the test run.

Name Type Description
additionalFileInfo

TestRunFileInfo[]

Additional supported files for the test run

configFileInfo

TestRunFileInfo

The load test YAML file that contains the the test configuration

inputArtifactsZipFileInfo

TestRunFileInfo

The zip file for all input artifacts

testScriptFileInfo

TestRunFileInfo

The test script file for the test run

userPropFileInfo

TestRunFileInfo

The user properties file

TestRunOutputArtifacts

The output artifacts for the test run.

Name Type Description
logsFileInfo

TestRunFileInfo

The test run report with metrics

resultFileInfo

TestRunFileInfo

The test run results file

TestRunStatistics

Test run statistics.

Name Type Description
errorCount

number (double)

Error count.

errorPct

number (double)

Error percentage.

maxResTime

number (double)

Max response time.

meanResTime

number (double)

Mean response time.

medianResTime

number (double)

Median response time.

minResTime

number (double)

Minimum response time.

pct1ResTime

number (double)

90 percentile response time.

pct2ResTime

number (double)

95 percentile response time.

pct3ResTime

number (double)

99 percentile response time.

receivedKBytesPerSec

number (double)

Received network bytes.

sampleCount

number (double)

Sampler count.

sentKBytesPerSec

number (double)

Send network bytes.

throughput

number (double)

Throughput.

transaction

string

Transaction name.

TestRunStatus

Test run status.

Value Description
ACCEPTED

Test run request is accepted.

NOTSTARTED

Test run is not yet started.

PROVISIONING

Test run is provisioning.

PROVISIONED

Test run is provisioned.

CONFIGURING

Test run is getting configured.

CONFIGURED

Test run configuration is done.

EXECUTING

Test run has started executing.

EXECUTED

Test run execution is completed.

DEPROVISIONING

Test run is getting deprovisioned.

DEPROVISIONED

Test run is deprovisioned.

DONE

Test run is completed.

CANCELLING

Test run is being cancelled.

CANCELLED

Test run request is cancelled.

FAILED

Test run request is failed.

VALIDATION_SUCCESS

Test run JMX file is validated.

VALIDATION_FAILURE

Test run JMX file validation is failed.