Customize cluster egress with a user-defined routing table in Azure Kubernetes Service (AKS)
You can customize the egress for your Azure Kubernetes Service (AKS) clusters to fit specific scenarios. AKS provisions a
Standard SKU load balancer for egress by default. However, the default setup may not meet the requirements of all scenarios if public IPs are disallowed or the scenario requires extra hops for egress.
This article walks through how to customize a cluster's egress route to support custom network scenarios. These scenarios include ones which disallow public IPs and require the cluster to sit behind a network virtual appliance (NVA).
- Azure CLI version 2.0.81 or greater. Run
az --versionto find the version. If you need to install or upgrade, see Install Azure CLI.
- API version
Requirements and limitations
Using outbound type is an advanced networking scenario and requires proper network configuration. The following requirements and limitations apply to using outbound type:
outboundTyperequires AKS clusters with a
outboundTypeto a value of
UDRrequires a user-defined route with valid outbound connectivity for the cluster.
outboundTypeto a value of
UDRimplies the ingress source IP routed to the load-balancer may not match the cluster's outgoing egress destination address.
Overview of customizing egress with a user-defined routing table
AKS doesn't automatically configure egress paths if
userDefinedRouting is set, which means you must configure the egress.
When you don't use standard load balancer (SLB) architecture, you must establish explicit egress. You must deploy your AKS cluster into an existing virtual network with a subnet that has been previously configured. This architecture requires explicitly sending egress traffic to an appliance like a firewall, gateway, or proxy, so a public IP assigned to the standard load balancer or appliance can handle the Network Address Translation (NAT).
Load balancer creation with
AKS clusters with an outbound type of UDR get a standard load balancer only when the first Kubernetes service of type
loadBalancer is deployed. The load balancer is configured with a public IP address for inbound requests and a backend pool for inbound requests. The Azure cloud provider configures inbound rules, but it doesn't configure outbound public IP address or outbound rules. Your UDR is the only source for egress traffic.
Azure load balancers don't incur a charge until a rule is placed.
Deploy a cluster with outbound type of UDR and Azure Firewall
To see an application of a cluster with outbound type using a user-defined route, see this restrict egress traffic with Azure firewall example.
Outbound type of UDR requires a route for 0.0.0.0/0 and a next hop destination of NVA in the route table. The route table already has a default 0.0.0.0/0 to the Internet. Without a public IP address for Azure to use for Source Network Address Translation (SNAT), simply adding this route won't provide you outbound Internet connectivity. AKS validates that you don't create a 0.0.0.0/0 route pointing to the Internet but instead to a gateway, NVA, etc. When using an outbound type of UDR, a load balancer public IP address for inbound requests isn't created unless you configure a service of type loadbalancer. AKS never creates a public IP address for outbound requests if you set an outbound type of UDR.
For more information on user-defined routes and Azure networking, see: