Share via

nestjs microservices using grpc to azure kubenertes using the LoadBalancer service

Shuchita Chiwande 0 Reputation points
2024-06-25T11:30:13.6866667+00:00

Hello, we have deployed a nestjs microservices using grpc to azure kubenertes using the LoadBalancer service method exposing a public IP from azure. The application itself is running and working, but sporadic we are getting the status code 14 unavailable read ECONNRESET. Normally the error can be reproduced if 5 or more minutes no request was sent to grpc server. Increasing the keep alive timeout hasnt solved this issue. Any idea what we can check? Currently there are running 2 pods on 2 nodes

Azure Container Registry
Azure Container Registry
An Azure service that provides a registry of Docker and Open Container Initiative images.
419 questions
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
1,950 questions
Microsoft Defender for Cloud
Microsoft Defender for Cloud
An Azure service that provides threat protection for workloads running in Azure, on-premises, and in other clouds. Previously known as Azure Security Center and Azure Defender.
1,250 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. kobulloc-MSFT 25,561 Reputation points Microsoft Employee
    2024-06-27T17:23:06.9833333+00:00

    Hello, @Shuchita Chiwande !

    Why am I receiving an error after 5 or more minutes of no requests sent when using a load balancer?

    Given that this works for the first couple minutes, it sounds like you are running into an idle timeout on your load balancer (default is 4 minutes):

    https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-tcp-idle-timeout?tabs=tcp-reset-idle-portal

    Azure Load Balancer rules have a default timeout range of 4 minutes to 100 minutes for Load Balancer rules, Outbound Rules, and Inbound NAT rules. The default setting is 4 minutes. If a period of inactivity is longer than the timeout value, there's no guarantee that the TCP or HTTP session is maintained between the client and your service.

    You can set the idle timeout and tcp reset for a load balancer by editing the load-balanced rule in the Azure portal:

    https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-tcp-idle-timeout?tabs=tcp-reset-idle-portal#set-tcp-reset-and-idle-timeout

    1. Sign in to the Azure portal.
    2. In the left-hand menu, select Resource groups.
    3. Select the resource group for your load balancer. In this example, the resource group is named myResourceGroup.
    4. Select your load balancer. In this example, the load balancer is named myLoadBalancer.
    5. In Settings, select Load balancing rules.
    6. Select your load-balancing rule. In this example, the load-balancing rule is named myLBrule.
    7. In the load-balancing rule, input your timeout value into Idle timeout (minutes).
    8. Under TCP reset, select Enabled.
    9. Select Save.

    Edit load balancer rules.

    Set idle timeout and tcp reset.

    I hope this has been helpful! Your feedback is important so please take a moment to accept answers.

    If you still have questions, please let us know what is needed in the comments so the question can be answered. Thank you for helping to improve Microsoft Q&A!

    User's image

    0 comments