App Service showing users intermittent 502 errors.

Question

Hi there,

I have a web app deployed on Azure App Service which is hosting a Django application.

Yesterday my users started reporting intermittent 502 errors when using the app. The users see the below.

I have gone through the Azure Troubleshooting guide here:
https://learn.microsoft.com/en-us/azure/app-service/troubleshoot-http-502-http-503

And have not arrived at a solution.

I can't see anything in the metrics that would indicate unusually high resource use. And I went through the Web App Down workflow which gave me the following:

I cannot find anything in the application logs to identify these errors (and I'm not even sure if these are the 502 that users are seeing). I have also tried sending HTTP Logs to a log analytics workspace and when I query it for 500 range errors I can see none.

Additional Context:
North Europe Region

Azure is currently carrying out scheduled maintenance on App service in the region. (unsure if related)

App Service Plan - P1v3, 1 app 1 slot

Other issues - We had our app service plan set to Automatic Autoscaling between 1 and 4 instances.

Starting on 11th April we started getting errors from the autoscaling:

Thinking this could be related I swapped temporarily over to a flat 2 instances when my users started reporting the 502 errors. They continued to occur. When I attempted to turn autoscaling back on my instances were marked as unhealthy and I got the 502 trying to visit the site so I put it back to manual 4 instances.

My leading theory right now is that perhaps this is performance related (though I can't see anything unusual in the app metrics) and that when autoscaling broke it was no longer scaling to meet demand and caused the 502s, then when I set it to manual 2 instances it still didn't meet demand. I'm waiting to hear from users if 4 instances has resolved the issue but obviously it's a very expensive short term solution. The site really doesn't have many users and 4 instances seems like overkill.

If anyone has any advice or options to gather more information it would be greatly appreciated.

Answer 1

@Brendan O'Brien Hi Brendan, this seems like there could be an underlying issue. Are you still experiencing this issue?

My concern is that your instances are unhealthy. Can you please try to scale up to a P2v3 and see if you encounter the same behavior? (Note that P2v3 cost more than P1v3) This should provide you with an entirely new instance(s) if there are health issues with any of your current instances.

Verify if your app is healthy again with your users. Assuming this resolved the issue, scale back down to P1V3 instance(s). This should have provided enough time to clear out the unhealthy instance(s) that you might be currently using.