Identify performance regressions by comparing test runs in Azure Load Testing
In this article, you'll learn how you can identify performance regressions by comparing test runs in the Azure Load Testing dashboard. The dashboard overlays the client-side and server-side metric graphs for each run, which allows you to quickly analyze performance issues.
You can compare load test runs for the following scenarios:
- Identify performance regressions between application builds or configurations. You could run a load test at each development sprint to ensure that the previous sprint didn't introduce performance issues.
- Identify which application component is responsible for a performance problem (root cause analysis). For example, an application redesign might result in slower application response times. Comparing load test runs might reveal that the root cause was a lack of database resources.
An Azure account with an active subscription. If you don't have an Azure subscription, create a free account before you begin.
An Azure Load Testing resource with a test plan that has multiple test runs. To create a Load Testing resource, see Create and run a load test.
Select test runs
To compare test runs in Azure Load Testing, you'll first have to select up to five runs within a load test. You can only compare runs that belong to the same load test.
A test run needs to be in the Done, Stopped, or Failed state to compare it.
Use the following steps to select the test runs:
Sign in to the Azure portal by using the credentials for your Azure subscription.
Go to your Azure Load Testing resource and then, on the left pane, select Tests.
You can also use the filters to find your load test.
Select the test whose runs you want to compare by selecting its name.
Select two or more test runs by selecting the corresponding checkboxes in the list.
You can choose a maximum of five test runs to compare.
Compare multiple test runs
After you've selected the test runs you want to compare, you can visually compare the client-side and server-side metrics for each test run in the load test dashboard.
Select the Compare button to open the load test dashboard.
Each test run is shown as an overlay in the different graphs.
Optionally, use the filters to customize the graphs.
The time filter is based on the duration of the tests. A value of zero indicates the start of the test, and the maximum value marks the duration of the longest test run.
Identify performance regressions
You can compare multiple test runs to identify performance regressions. For example, before deploying a new application version in production, you can verify that the performance hasn't degraded.
Use the client-side metrics, such as requests per second or response time, on the load test dashboard to quickly spot performance changes between different load test runs:
Hover over the client-side metrics graphs to compare the values across the different test runs.
In the following screenshot, you notice that the metric values for Requests per second and Response Time are significantly different. This difference indicates that the application performance dropped significantly between the two test runs.
Optionally, use the Requests filter to compare a specific application request in the JMeter script.
Identify the root cause
When there's a performance issue, you can use the server-side metrics to analyze what the root cause of the problem is. Azure Load Testing can capture server-side resource metrics for Azure-hosted applications.
Hover over the server-side metrics graphs to compare the values across the different test runs.
In the following screenshot, you notice from the Response time and Requests that the application performance has degraded. You can also see that for one test run, the database RU Consumption peeks at 100%. This indicates that the root cause is likely the database Provisioned Throughput.
Optionally, select Configure metrics to add or remove server-side metrics.
You can add more server-side metrics for the selected Azure app components to further investigate performance problems. The dashboard immediately shows the additional metrics data, and you don't have to rerun the load test.
Optionally, use the Resource filter to hide or show all metric graphs for an Azure component.
- Learn more about exporting the load test results for reporting.
- Learn more about troubleshooting load test execution errors.
- Learn more about configuring automated performance testing with CI/CD.