Edit

Configure kube-proxy in Azure Kubernetes Service (AKS) (preview)

  • Article
  • 3 minutes to read

kube-proxy is a component of Kubernetes that handles routing traffic for services within the cluster. There are two backends available for Layer 3/4 load balancing in upstream kube-proxy - iptables and IPVS.

  • iptables is the default backend utilized in the majority of Kubernetes clusters. It is simple and well supported, but is not as efficient or intelligent as IPVS.
  • IPVS utilizes the Linux Virtual Server, a layer 3/4 load balancer built into the Linux kernel. IPVS provides a number of advantages over the default iptables configuration, including state awareness, connection tracking, and more intelligent load balancing.

The AKS managed kube-proxy DaemonSet can also be disabled entirely if that is desired to support bring-your-own CNI.

Prerequisites

  • Azure CLI with aks-preview extension 0.5.105 or later.
  • If using ARM or the REST API, the AKS API version must be 2022-08-02-preview or later.

Install the aks-preview Azure CLI extension

Important

AKS preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. AKS previews are partially covered by customer support on a best-effort basis. As such, these features aren't meant for production use. For more information, see the following support articles:

To install the aks-preview extension, run the following command:

az extension add --name aks-preview

Run the following command to update to the latest version of the extension released:

az extension update --name aks-preview

Register the 'KubeProxyConfigurationPreview' feature flag

Register the KubeProxyConfigurationPreview feature flag by using the az feature register command, as shown in the following example:

az feature register --namespace "Microsoft.ContainerService" --name "KubeProxyConfigurationPreview"

It takes a few minutes for the status to show Registered. Verify the registration status by using the az feature show command:

az feature show --namespace "Microsoft.ContainerService" --name "KubeProxyConfigurationPreview"

When the status reflects Registered, refresh the registration of the Microsoft.ContainerService resource provider by using the az provider register command:

az provider register --namespace Microsoft.ContainerService

Configurable options

The full kube-proxy configuration structure can be found in the AKS Cluster Schema.

  • enabled - whether or not to deploy the kube-proxy DaemonSet. Defaults to true.
  • mode - can be set to IPTABLES or IPVS. Defaults to IPTABLES.
  • ipvsConfig - if mode is IPVS, this object contains IPVS-specific configuration properties.
    • scheduler - which connection scheduler to utilize. Supported values:
      • LeastConnections - sends connections to the backend pod with the fewest connections
      • RoundRobin - distributes connections evenly between backend pods
    • tcpFinTimeoutSeconds - the value used for timeout after a FIN has been received in a TCP session
    • tcpTimeoutSeconds - the value used for timeout length for idle TCP sessions
    • udpTimeoutSeconds - the value used for timeout length for idle UDP sessions

Note

IPVS load balancing operates in each node independently and is still only aware of connections flowing through the local node. This means that while LeastConnections results in more even load under higher number of connections, when low numbers of connections (# connects < 2 * node count) occur traffic may still be relatively unbalanced.

Utilize kube-proxy configuration in a new or existing AKS cluster using Azure CLI

kube-proxy configuration is a cluster-wide setting. No action is needed to update your services.

Warning

Changing the kube-proxy configuration may cause a slight interruption in cluster service traffic flow.

To begin, create a JSON configuration file with the desired settings:

Create a configuration file

{
  "enabled": true,
  "mode": "IPVS",
  "ipvsConfig": {
    "scheduler": "LeastConnection",
    "TCPTimeoutSeconds": 900,
    "TCPFINTimeoutSeconds": 120,
    "UDPTimeoutSeconds": 300
  }
}

Deploy a new cluster

Deploy your cluster using az aks create and pass in the configuration file:

az aks create -g <resourceGroup> -n <clusterName> --kube-proxy-config kube-proxy.json

Update an existing cluster

Configure your cluster using az aks update and pass in the configuration file:

az aks update -g <resourceGroup> -n <clusterName> --kube-proxy-config kube-proxy.json

Limitations

When using kube-proxy IPVS, the following restrictions apply:

  • Azure Network Policy is not supported.

Next steps

Learn more about utilizing the Standard Load Balancer for inbound traffic at the AKS Standard Load Balancer documentation.

Learn more about using Internal Load Balancer for Inbound traffic at the AKS Internal Load Balancer documentation.

Learn more about Kubernetes services at the Kubernetes services documentation.