Customize cluster egress with outbound types in Azure Kubernetes Service (AKS)
You can customize egress for an AKS cluster to fit specific scenarios. By default, AKS will provision a standard SKU load balancer to be set up and used for egress. However, the default setup may not meet the requirements of all scenarios if public IPs are disallowed or additional hops are required for egress.
This article covers the various types of outbound connectivity that are available in AKS clusters.
You can now update the
outboundType after cluster creation. This feature is in preview. See Updating `outboundType after cluster creation (preview).
outboundTyperequires AKS clusters with a
Outbound types in AKS
You can configure an AKS cluster using the following outbound types: load balancer, NAT gateway, or user-defined routing. The outbound type impacts only the egress traffic of your cluster. For more information, see setting up ingress controllers.
You can use your own [route table][byo-route-table] with UDR and kubenet networking. Make sure your cluster identity (service principal or managed identity) has Contributor permissions to the custom route table.
Outbound type of
The load balancer is used for egress through an AKS-assigned public IP. An outbound type of
loadBalancer supports Kubernetes services of type
loadBalancer, which expect egress out of the load balancer created by the AKS resource provider.
loadBalancer is set, AKS automatically completes the following configuration:
- A public IP address is provisioned for cluster egress.
- The public IP address is assigned to the load balancer resource.
- Backend pools for the load balancer are set up for agent nodes in the cluster.
For more information, see using a standard load balancer in AKS.
Outbound type of
userAssignedNatGateway are selected for
outboundType, AKS relies on Azure Networking NAT gateway for cluster egress.
managedNatGatewaywhen using managed virtual networks. AKS will provision a NAT gateway and attach it to the cluster subnet.
userAssignedNatGatewaywhen using bring-your-own virtual networking. This option requires that you have provisioned a NAT gateway before cluster creation.
For more information, see using NAT gateway with AKS.
Outbound type of
userDefinedRouting outbound type is an advanced networking scenario and requires proper network configuration.
userDefinedRouting is set, AKS won't automatically configure egress paths. The egress setup must be done by you.
You must deploy the AKS cluster into an existing virtual network with a subnet that has been previously configured. Since you're not using a standard load balancer (SLB) architecture, you must establish explicit egress. This architecture requires explicitly sending egress traffic to an appliance like a firewall, gateway, proxy or to allow NAT to be done by a public IP assigned to the standard load balancer or appliance.
For more information, see configuring cluster egress via user-defined routing.
outboundType after cluster creation (preview)
Changing the outbound type after cluster creation will deploy or remove resources as required to put the cluster into the new egress configuration.
The following tables show the supported migration paths between outbound types for managed and BYO virtual networks.
Supported Migration Paths for Managed VNet
|userAssignedNATGateway||Not Supported||Not Supported||N/A||Not Supported|
Supported Migration Paths for BYO VNet
|managedNATGateway||Not Supported||N/A||Not Supported||Not Supported|
Migration is only supported between
managedNATGateway (if using a managed virtual network),
userDefinedRouting (if using a custom virtual network).
Migrating the outbound type to user managed types (
userDefinedRouting) will change the outbound public IP addresses of the cluster.
if Authorized IP ranges is enabled, please make sure new outbound ip range is appended to authorized ip range.
Changing the outbound type on a cluster is disruptive to network connectivity and will result in a change of the cluster's egress IP address. If any firewall rules have been configured to restrict traffic from the cluster, you need to update them to match the new egress IP address.
AKS preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. AKS previews are partially covered by customer support on a best-effort basis. As such, these features aren't meant for production use. For more information, see the following support articles:
aks-preview Azure CLI extension
aks-preview version 0.5.113 is required.
- Install and update the
# Install aks-preview extension az extension add --name aks-preview # Update aks-preview extension az extension update --name aks-preview
AKS-OutBoundTypeMigrationPreview feature flag
- Register the
AKS-OutBoundTypeMigrationPreviewfeature flag using the
az feature registercommand. It takes a few minutes for the status to show Registered.
az feature register --namespace "Microsoft.ContainerService" --name "AKS-OutBoundTypeMigrationPreview"
- Verify the registration status using the
az feature showcommand.
az feature show --namespace "Microsoft.ContainerService" --name "AKS-OutBoundTypeMigrationPreview"
- When the status reflects Registered, refresh the registration of the Microsoft.ContainerService resource provider using the
az provider registercommand.
az provider register --namespace Microsoft.ContainerService
Update cluster to use a new outbound type
- Update the outbound configuration of your cluster using the
az aks updatecommand.
Update cluster from loadbalancer to managedNATGateway
az aks update -g <resourceGroup> -n <clusterName> --outbound-type managedNATGateway --nat-gateway-managed-outbound-ip-count <number of managed outbound ip>
Update cluster from managedNATGateway to loadbalancer
az aks update -g <resourceGroup> -n <clusterName> \ --outbound-type loadBalancer \ <--load-balancer-managed-outbound-ip-count <number of managed outbound ip>| --load-balancer-outbound-ips <outbound ip ids> | --load-balancer-outbound-ip-prefixes <outbound ip prefix ids> >
Do not reuse an IP address that is already in use in prior outbound configurations.
Update cluster from managedNATGateway to userDefinedRouting
- Add route
0.0.0.0/0to default route table. Please refer to Customize cluster egress with a user-defined routing table in Azure Kubernetes Service (AKS)
az aks update -g <resourceGroup> -n <clusterName> --outbound-type userDefinedRouting
Update cluster from loadbalancer to userAssignedNATGateway in BYO vnet scenario
- Associate nat gateway with subnet where the workload is associated with. Please refer to Create a managed or user-assigned NAT gateway
az aks update -g <resourceGroup> -n <clusterName> --outbound-type userAssignedNATGateway