Troubleshooting application gateway timeout errors caused by excessive load

Question

We are experiencing timeout errors when calling an API through an application gateway. The connection fails and returns a 500 error exactly after 2.2 minutes for most services. We noticed that the issue occurs when we generate heavy requests per second (around 40-50 transactions per second) but improves when we hit the system with lower load (around 20-30 transactions per sec). We suspect that the issue lies with the recently implemented Application Gateway. Can someone who works on Application Gateway help us troubleshoot this issue?

Answer 1

@Anonymous

Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well.

I understand that you are experiencing HTTP 500 Response when you are increasing the load on the system.

May I ask how you are confirming the issue is with Load Balancing

• However, it could be possible that the backend is incapable of handling the request?
• Can you check the access log of one such HTTP 500 response and see what the backend response was?
  • Check for Access Logs and see what is the value of "serverStatus"
    • 
• You can also increase the Number of instances in case you are using Manual scaling.

Cheers,

Kapil