Configure Static Egress Gateway in Azure Kubernetes Service (AKS)

Static Egress Gateway in AKS provides a streamlined solution for configuring fixed source IP addresses for outbound traffic from your AKS workloads. This feature allows you to route egress traffic through a dedicated "gateway node pool". By using the Static Egress Gateway, you can efficiently manage and control outbound IP addresses and ensure that your AKS workloads can communicate with external systems securely and consistently, using predefined IPs.

This article provides step-by-step instructions to set up a Static Egress Gateway node pool in your AKS cluster, enabling you to configure fixed source IP addresses for outbound traffic from your Kubernetes workloads.

Important

AKS preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. AKS previews are partially covered by customer support on a best-effort basis. As such, these features aren't meant for production use. For more information, see the following support articles:

Limitations and considerations

  • Static Egress Gateway isn't supported in clusters with Azure CNI Pod Subnet.

  • Kubernetes network policies won't apply to traffic leaving the cluster through the gateway node pool.

    • This shouldn't affect cluster traffic control as only egress traffic from annotated pods routed to the gateway node pool are affected.
  • The gateway node pool isn't intended for general-purpose workloads and should be used for egress traffic only.

  • Windows node pools can't be used as gateway node pools.

  • hostNetwork pods cannot be annotated to use the gateway node pool.

Before you begin

Install the aks-preview Azure CLI extension

  1. Install the aks-preview extension using the az extension add command.

    az extension add --name aks-preview
    
  2. Update to the latest version of the extension using the az extension update command.

    az extension update --name aks-preview
    

Register the StaticEgressGatewayPreview feature flag

  1. Register the StaticEgressGatewayPreview feature flag using the az feature register command.

    az feature register --namespace "Microsoft.ContainerService" --name "StaticEgressGatewayPreview"
    

    It takes a few minutes for the status to show Registered.

  2. Verify the registration status using the az feature show command.

    az feature show --namespace "Microsoft.ContainerService" --name "StaticEgressGatewayPreview"
    
  3. When the status reflects Registered, refresh the registration of the Microsoft.ContainerService resource provider using the az provider register command.

    az provider register --namespace Microsoft.ContainerService
    

Create or update an AKS cluster with Static Egress Gateway

Before you can create and manage gateway node pools, you must enable the Static Egress Gateway feature for your AKS cluster. You can do this when creating a new cluster or by updating an existing cluster using az aks update.

az aks create -n <cluster-name> -g <resource-group> --enable-static-egress-gateway

Create a Gateway Node pool

After enabling the feature, create a dedicated gateway node pool. This node pool handles the egress traffic through the specified public IP prefix. The --gateway-prefix-size is the size of the public IP prefix to be applied to the gateway node pool nodes. The allowed range is 28-31.

az aks nodepool add --cluster-name <cluster-name> \
    --name <nodepool-name> \
    --resource-group <resource-group> \
    --mode gateway \
    --node-count <number-of-nodes> \
    --gateway-prefix-size <prefix-size>

Note

  • The number of nodes must fit within the capacity allowed by the selected prefix size. For example, a /30 prefix supports up to 4 nodes, and at least 2 nodes are required for high availability. Since you can’t adjust the node count dynamically, plan your nodes according to the fixed limit set by the prefix size.
  • You can define the SKU of the VM to use in your gateway node pool with the --vm-size parameter. You should understand your specific needs and plan accordingly to ensure the right performance and cost balance.

Scale the Gateway Node pool (Optional)

If necessary, you can resize the gateway node pool within the limits defined by the prefix size but it doesn't support autoscaling.

az aks nodepool scale --cluster-name <cluster-name> -n <nodepool-name> --node-count <desired-node-count>

Create a Static Gateway Configuration

Define the gateway configuration by creating a StaticGatewayConfiguration custom resource. This configuration specifies which node pool and public IP prefix to use.

apiVersion: egressgateway.kubernetes.azure.com/v1alpha1
kind: StaticGatewayConfiguration
metadata:
  name: <gateway-config-name>
  namespace: <namespace>
spec:
  gatewayNodepoolName: <nodepool-name>
  excludeCidrs:  # Optional
  - 10.0.0.0/8
  - 172.16.0.0/12
  - 169.254.169.254/32
  publicIpPrefixId: /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Network/publicIPPrefixes/<prefix-name> # Optional

Tip

If you don't set publicIpPrefixId, a public IP prefix will be created for you automatically. When running kubectl describe StaticGatewayConfiguration <gateway-config-name> -n <namespace>, you can see the "Egress Ip Prefix" in the status. This is the newly created public IP prefix. You can also use an existing public IP prefix by specifying its resource ID in the publicIpPrefixId argument. You need to grant "Network Contributor" role to AKS cluster's identity in this case.

Annotate Pods to Use the Gateway Configuration

To route traffic from specific pods through the gateway node pool, annotate the pod template in the deployment configuration.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: <deployment-name>
  namespace: <namespace>
spec:
  template:
    metadata:
      annotations:
        kubernetes.azure.com/static-gateway-configuration: <gateway-config-name>

Note

The CNI plugin on each node will automatically configure the pod to route its traffic through the selected gateway nodepool.

Monitor and Manage Gateway Configurations

Once deployed, you can monitor the status of your gateway configurations through the AKS cluster. The status section in the StaticGatewayConfiguration resource is updated with details such as assigned IPs and WireGuard configurations.

Delete a Gateway Node pool (Optional)

To remove a gateway node pool, ensure all associated configurations are appropriately handled before deletion.

az aks nodepool delete --cluster-name <cluster-name> -n <nodepool-name>

Disable the Static Egress Gateway Feature (Optional)

If you no longer need the Static Egress Gateway, you can disable the feature and uninstall the operator. Ensure all gateway node pools are deleted first.

az aks update -n <cluster-name> -g <resource-group> --disable-static-egress-gateway

By following these steps, you can effectively set up and manage Static Egress Gateway configurations in your AKS cluster, enabling controlled and consistent egress traffic from your workloads.