Service Disruption during rolling update of kubernetes cluster

Question

Hi Team,

  We are running self managed Kubernetes on Microsoft azure Infrastructure and are facing disruption in service while running rolling update.

  User Case:
          We are having a user facing service which is exposed using Azure application gateway backed by Kubernetes service of type NodePort.
          The application gateways is configured using terraform and uses a static port pointing to the Kubernetes NodePort service, 
          and we have 3 VMSS in the application gateway backend pool. 

  Issue: 
      Customer faces some disruption in service during rolling update.

  Scenario:
        While running the rolling update, we cordon the and drain the node.
       As we are using Kubernetes NodePort service even cordoning does not help as the application gateway will not know the done is not in the 
       Service until the probe fails which is configured for 30 sec as timeout and unhealthy threshold set to 3. 


       Any suggestion how we can fix this issue? 
       We would also like to know how this scenario is handled in AKS.

Answer 1

@Balgi, Sheshgiri AKS supports native integration with Application Gateway using an ingress controller. This is going to be much easier to manage and simplify Terraform configuration.

https://learn.microsoft.com/en-us/azure/application-gateway/ingress-controller-overview

You can also use App GW with an internal load balancer instead of using nodeport.

https://learn.microsoft.com/en-us/azure/application-gateway/application-gateway-ilb-arm

https://learn.microsoft.com/en-us/azure/aks/internal-lb