Azure App Service plan is not evenly distributing traffic across instances

Question

We are seeing that Azure App Service (V3 P2v3 x 6 Instances) is not evenly distributing requests across the 6 instances.

The result is that one instance is getting swamped with heavy requests and is failing because of the heavy load, meanwhile other instances are having it easy. (We do not have session persistance configured)

As stated here, the Layer 7 round robin loadbalancer that the App service plan uses, is a Microsoft managed service and it seems that we have no control over it. https://learn.microsoft.com/en-us/answers/questions/758546/azure-app-serivce-load-balancer

Is there another solution or service we should be looking at here?

Thx in advance.

Answer 1

Thanks for posting this question.

ARR affinity is a feature that enables sticky sessions' by ensuring that subsequent requests from a client are routed to the same instance of a web app. Disabling ARR Affinity assumes that your application is either stateless, or the session state is stored on a remote service such as a cache or database.
(We do not have session persistence configured) - Just to clarify, have you disabled ARR cookies?

Just to highlight from my answer here : Azure WebApps by default have ARR Affinity cookie enabled, this cookie pairs a client request to a specific server. However, Azure Web Apps is a stateless platform and, in an environment, where we are scaling the Website across multiple instances, the ARR Affinity cookie will be bound to a specific server. In case, the server is no longer in rotation, then all the requests corresponding to the ARR Affinity cookie will fail. It’s advisable to avoid the use ARR cookies in a scaled environment where we have multiple instances that serve our application requests. Disabling ARR cookies is a sustainable resolution for issues related to ARR Affinity cookies in scaled environments, where these cookies rely on the relationship with the worker machine they are paired with.

Also, when we have ARR Affinity cookies enabled, we are truly not utilizing the scaling ability where a requests can get handled by any instance that has the resources to handle the request. Instead, based on ARR Affinity cookie mapping, the requests will always go to only to the server tied to the Affinity cookie.

Kindly checkout this doc: The Ultimate Guide to Running Healthy Apps in the Cloud