Events
Mar 17, 11 PM - Mar 21, 11 PM
Join the meetup series to build scalable AI solutions based on real-world use cases with fellow developers and experts.
Register nowThis browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Kubernetes uses Container Networking Interface (CNI) plugins to manage networking in Kubernetes clusters. CNIs are responsible for assigning IP addresses to pods, network routing between pods, Kubernetes Service routing, and more.
AKS provides multiple CNI plugins you can use in your clusters depending on your networking requirements.
Choosing a CNI plugin for your AKS cluster largely depends on which networking model fits your needs best. Each model has its own advantages and disadvantages you should consider when planning your AKS cluster.
AKS uses two main networking models: overlay network and flat network.
Both networking models have multiple supported CNI plugin options. The main differences between the models are how pod IP addresses are assigned and how traffic leaves the cluster.
Overlay networking is the most common networking model used in Kubernetes. In overlay networks, pods are given an IP address from a private, logically separate CIDR from the Azure VNet subnet where AKS nodes are deployed. This allows for simpler and often better scalability than the flat network model.
In overlay networks, pods can communicate with each other directly. Traffic leaving the cluster is Source Network Address Translated (SNAT'd) to the node's IP address, and inbound Pod IP traffic is routed through some service, such as a load balancer. This means that the pod IP address is "hidden" behind the node's IP address. This approach reduces the number of VNet IP addresses required for your clusters.
Azure Kubernetes Service provides the following CNI plugins for overlay networking:
Unlike an overlay network, a flat network model in AKS assigns IP addresses to pods from a subnet from the same Azure VNet as the AKS nodes. This means that traffic leaving your clusters is not SNAT'd, and the pod IP address is directly exposed to the destination. This can be useful for some scenarios, such as when you need to expose pod IP addresses to external services.
Azure Kubernetes Service provides two CNI plugins for flat networking. This article doesn't go into depth for each plugin option. For more information, see the linked documentation:
When choosing a CNI, there are several factors to consider. Each networking model has its own advantages and disadvantages, and the best choice for your cluster will depend on your specific requirements.
The two main networking models in AKS are overlay and flat networks.
Overlay networks:
Flat networks:
When choosing a networking model, consider the use cases for each CNI plugin and the type of network model it uses:
CNI plugin | Networking model | Use case highlights |
---|---|---|
Azure CNI Overlay | Overlay | - Best for VNET IP conservation - Max node count supported by API Server + 250 pods per node - Simpler configuration -No direct external pod IP access |
Azure CNI Pod Subnet | Flat | - Direct external pod access - Modes for efficient VNet IP usage or large cluster scale support(Preview) |
Kubenet (Legacy) | Overlay | - Prioritizes IP conservation - Limited scale - Manual route management |
Azure CNI Node Subnet (Legacy) | Flat | - Direct external pod access - Simpler configuration - Limited scale - Inefficient use of VNet IPs |
You might also want to compare the features of each CNI plugin. The following table provides a high-level comparison of the features supported by each CNI plugin:
Feature | Azure CNI Overlay | Azure CNI Pod Subnet | Azure CNI Node Subnet (Legacy) | Kubenet |
---|---|---|---|---|
Deploy cluster in existing or new VNet | Supported | Supported | Supported | Supported - manual UDRs |
Pod-VM connectivity with VM in same or peered VNet | Pod initiated | Both ways | Both ways | Pod initiated |
On-premises access via VPN/Express Route | Pod initiated | Both ways | Both ways | Pod initiated |
Access to service endpoints | Supported | Supported | Supported | Supported |
Expose services using load balancer | Supported | Supported | Supported | Supported |
Expose services using App Gateway | Currently not supported | Supported | Supported | Supported |
Expose services using ingress controller | Supported | Supported | Supported | Supported |
Windows node pools | Supported | Supported | Supported | Not supported |
Default Azure DNS and Private Zones | Supported | Supported | Supported | Supported |
VNet Subnet sharing across multiple clusters | Supported | Supported | Supported | Not supported |
Depending on the CNI you use, your cluster virtual network resources can be deployed in one of the following ways:
Although capabilities like service endpoints or UDRs are supported, the support policies for AKS define what changes you can make. For example:
There are several requirements and considerations to keep in mind when planning your network configuration for AKS:
169.254.0.0/16
, 172.30.0.0/16
, 172.31.0.0/16
, or 192.0.2.0/24
for the Kubernetes service address range, pod address range, or cluster virtual network address range.Microsoft.Network/virtualNetworks/subnets/join/action
Microsoft.Authorization/roleAssignments/write
Microsoft.Network/virtualNetworks/subnets/read
(only needed if you are defining your own subnets and CIDRs)Azure Kubernetes Service feedback
Azure Kubernetes Service is an open source project. Select a link to provide feedback:
Events
Mar 17, 11 PM - Mar 21, 11 PM
Join the meetup series to build scalable AI solutions based on real-world use cases with fellow developers and experts.
Register nowTraining
Module
Choose the best networking plugin for AKS - Training
Learn how to choose between Azure CNI and kubenet, two networking plugin options for an Azure Kubernetes Service cluster.
Certification
Microsoft Certified: Azure Network Engineer Associate - Certifications
Demonstrate the design, implementation, and maintenance of Azure networking infrastructure, load balancing traffic, network routing, and more.