Edit

Configure Azure CNI Powered by Cilium in Azure Kubernetes Service (AKS) (Preview)

  • Article
  • 4 minutes to read

Azure CNI Powered by Cilium combines the robust control plane of Azure CNI with the dataplane of Cilium to provide high-performance networking and security.

By making use of eBPF programs loaded into the Linux kernel and a more efficient API object structure, Azure CNI Powered by Cilium provides the following benefits:

  • Functionality equivalent to existing Azure CNI and Azure CNI Overlay plugins
  • Faster service routing
  • More efficient network policy enforcement
  • Better observability of cluster traffic
  • Support for larger clusters (more nodes, pods, and services)

IP Address Management (IPAM) with Azure CNI Powered by Cilium

Azure CNI Powered by Cilium can be deployed using two different methods for assigning pod IPs:

  • assign IP addresses from a VNet (similar to existing Azure CNI with Dynamic Pod IP Assignment)
  • assign IP addresses from an overlay network (similar to Azure CNI Overlay mode)

Note

Azure CNI Overlay networking currently requires the Microsoft.ContainerService/AzureOverlayPreview feature and may be available only in certain regions. For more information, see Azure CNI Overlay networking.

If you aren't sure which option to select, read "Choosing a network model to use".

Network Policy Enforcement

Cilium enforces network policies to allow or deny traffic between pods. With Cilium, you don't need to install a separate network policy engine such as Azure Network Policy Manager or Calico.

Limitations

Azure CNI powered by Cilium currently has the following limitations:

  • Available only for new clusters.
  • Available only for Linux and not for Windows.
  • Cilium L7 policy enforcement is disabled.
  • Hubble is disabled.
  • Kubernetes services with internalTrafficPolicy=Local aren't supported (Cilium issue #17796).
  • Multiple Kubernetes services can't use the same host port with different protocols (for example, TCP or UDP) (Cilium issue #14287).
  • Network policies may be enforced on reply packets when a pod connects to itself via service cluster IP (Cilium issue #19406).

Prerequisites

  • Azure CLI version 2.41.0 or later. Run az --version to see the currently installed version. If you need to install or upgrade, see [Install Azure CLI][/cli/azure/install-azure-cli].
  • Azure CLI with aks-preview extension 0.5.109 or later.
  • If using ARM templates or the REST API, the AKS API version must be 2022-09-02-preview or later.

Install the aks-preview Azure CLI extension

Important

AKS preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. AKS previews are partially covered by customer support on a best-effort basis. As such, these features aren't meant for production use. For more information, see the following support articles:

To install the aks-preview extension, run the following command:

az extension add --name aks-preview

Run the following command to update to the latest version of the extension released:

az extension update --name aks-preview

Register the 'CiliumDataplanePreview' feature flag

Register the CiliumDataplanePreview feature flag by using the az feature register command, as shown in the following example:

az feature register --namespace "Microsoft.ContainerService" --name "CiliumDataplanePreview"

It takes a few minutes for the status to show Registered. Verify the registration status by using the az feature show command:

az feature show --namespace "Microsoft.ContainerService" --name "CiliumDataplanePreview"

When the status reflects Registered, refresh the registration of the Microsoft.ContainerService resource provider by using the az provider register command:

az provider register --namespace Microsoft.ContainerService

Create a new AKS Cluster with Azure CNI Powered by Cilium

Option 1: Assign IP addresses from a VNet

Run the following commands to create a resource group and VNet with a subnet for nodes and a subnet for pods.

# Create the resource group
az group create --name <resourceGroupName> --location <location>

# Create a VNet with a subnet for nodes and a subnet for pods
az network vnet create -g <resourceGroupName> --location <location> --name <vnetName> --address-prefixes <address prefix, example: 10.0.0.0/8> -o none 
az network vnet subnet create -g <resourceGroupName> --vnet-name <vnetName> --name nodesubnet --address-prefixes <address prefix, example: 10.240.0.0/16> -o none 
az network vnet subnet create -g <resourceGroupName> --vnet-name <vnetName> --name podsubnet --address-prefixes <address prefix, example: 10.241.0.0/16> -o none

Create the cluster using --enable-cilium-dataplane:

az aks create -n <clusterName> -g <resourceGroupName> -l <location> \
  --max-pods 250 \
  --network-plugin azure \
  --vnet-subnet-id /subscriptions/<subscriptionId>/resourceGroups/<resourceGroupName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/nodesubnet \
  --pod-subnet-id /subscriptions/<subscriptionId>/resourceGroups/<resourceGroupName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/podsubnet \
  --enable-cilium-dataplane

Option 2: Assign IP addresses from an overlay network

Run this commands to create a cluster with an overlay network and Cilium. Replace the values for <clusterName>, <resourceGroupName>, and <location>:

az aks create -n <clusterName> -g <resourceGroupName> -l <location> \
  --network-plugin azure \
  --network-plugin-mode overlay \
  --pod-cidr 192.168.0.0/16 \
  --enable-cilium-dataplane

Frequently asked questions

  • Can I customize Cilium configuration?

    No, the Cilium configuration is managed by AKS and can't be modified. We recommend that customers who require more control use AKS BYO CNI and install Cilium manually.

  • Can I use CiliumNetworkPolicy custom resources instead of Kubernetes NetworkPolicy resources?

    CiliumNetworkPolicy custom resources aren't officially supported. We recommend that customers use Kubernetes NetworkPolicy resources to configure network policies.

Next steps

Learn more about networking in AKS in the following articles: