I need to do automated performance tests of new releases of a web application. The purpose of the tests is primarily to catch performance regressions before a new release goes live.
The production environment isn't in the cloud but we have test environments running on Azure VMs.
We're considering running our tests on Azure VMs but we're concerned that the VM performance is too variable to allow us to measure changes in the application.
Does anyone know how stable Azure VM (mostly cpu but also disks, etc.) performance is hour-to-hour/day-to-day/month-to-month? Am I likely to have difficulties comparing the performance of multiple test runs on the same VMs?