Timeout issue with requests in Azure Virtual Machines

Question

We currently have a virtual environment in Azure with two machines, machines A and B. Machine A has a web service running on port 6443 and a test client written in python, while machine B only has the same test python client script.

The test script does a direct request the 6443 endpoint on the server.  The server endpoint is supposed to return a response to the calling client when a specific task is finished.  The task takes typically everything from five minutes to an hour depending on the definition of the task.

When the task takes five minutes it returns the call after the same five minutes and the test script finishes, both when tested from the machine A (where the webervice runs), and the machine B that has to call the webservice over the virtual network to machine A.

When the task takes more than 30 minutes the client does not seem to get any response from the server when the test is run from the machine B that has to call the service over the virtual network.  The server A actually finishes the task and presumably returns an answer, but the client application does not seem to receive any response – it is lost somewhere.  When the test client is ran on the server A – the same machine that runs the webservice, and no traffic is on the virtual network the communication seems to be ok.

So the question then is: are there any timeout mechanisms that stalls/kills calls that takes longer than 30 minutes?

Answer 1

Hi Live, welcome to MS Q&A

Yes, there are several potential timeout mechanisms in Azure and other layers of your setup that could affect long-running HTTP requests. Here’s a detailed approach to diagnosing and addressing the issue:

1. Client-Side Timeout

Ensure that the client script running on Machine B does not have a timeout that is less than the duration your task takes. In Python's requests library, the timeout parameter controls

Solution:

Increase the timeout setting in the client script

import requests

response = requests.get('https://<MachineA_IP_or_DNS>:6443', timeout=3600)  # timeout in seconds

2. Azure Load Balancer/Network Timeouts

Azure Load Balancer rules have a default timeout range of 4 minutes to 100 minutes for Load Balancer rules, Outbound Rules, and Inbound NAT rules. The default setting is 4 minutes. If a period of inactivity is longer than the timeout value, there's no guarantee that the TCP or HTTP session is maintained between the client and your service.

Solution:

Increase the idle timeout for your Azure Load Balancer, check this doc for step wise solution https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-tcp-idle-timeout?tabs=tcp-reset-idle-portal

3. Application Gateway Timeouts

If you're using Azure Application Gateway, The TCP idle timeout is a 4-minute default on the frontend virtual IP (VIP) of both v1 and v2 SKU of Application Gateway. You can configure the TCP idle timeout value on v1 and v2 Application Gateways to be anywhere between 4 minutes and 30 minutes.

Solution - please check this thread https://learn.microsoft.com/en-us/answers/questions/1371233/getting-gateway-timeout-in-azure-application-gatew

Please let me know if any further questions

Kindly accept answer, if it helps

Thanks
Deepanshu