Adding dynamic pod ip allocation to existing cluster is not working.

Question

We have a set of 2 clusters running on Kubernetes 1.34.2

We decided to enable dynamic pod allocation, we use terraform to manage our cluster.

When we are adding the pod subnet into the node pool we are not able to do it.

We have 3 node pools, admin, applications and spot.

We have existing workload there, if we try to add a temp app node pool to migrate our traffic, we can't

"All or none of the agentpools should set podsubnet"

But we can't also add pod subnet to all existing pools, even the admin that have a default mechanism to ensure the migration process.

Because when is creating the temp admin one is facing:

"All or none of the agentpools should set podsubnet"

Is there a path or safe way to do the migration?

Answer 1

Dynamic pod IP allocation (Azure CNI Pod Subnet - Dynamic IP Allocation) and static block allocation (Azure CNI Pod Subnet - Static Block Allocation) both require that all agent pools in a cluster follow the same CNI mode and that pod subnet usage is consistent across pools.

From the documented behavior:

• Only one CNI type can be used per cluster: traditional Azure CNI vs Azure CNI Pod Subnet (dynamic or static). Mixed CNI modes in the same cluster are not supported.
• For Azure CNI Pod Subnet modes, only a single pod subnet can be assigned to a given node pool, but multiple node pools or clusters can share that same pod subnet.
• Static Block vs Dynamic IP allocation is controlled per node pool via --pod-ip-allocation-mode, but a single subnet cannot be used in both modes at the same time.
• For static block allocation, migration or updating existing node pools in-place is explicitly not supported; only adding new node pools with a different subnet is supported.

The error All or none of the agentpools should set podsubnet is consistent with the requirement that the cluster’s agent pools be configured consistently with respect to pod subnet usage. In practice this means there is no supported partial/gradual enablement of Azure CNI Pod Subnet on only some pools while others remain without a pod subnet.

The documented migration pattern is:

1. Create a new subnet in the existing VNet dedicated to the new mode (for example, Static Block Allocation) and size it according to the IP planning guidance.
2. Add a new agent pool to the existing cluster that uses the new pod subnet and the desired allocation mode (DynamicIndividual for dynamic, StaticBlock for static), referencing:
   • --vnet-subnet-id for the node subnet
   • --pod-subnet-id for the pod subnet
   • --pod-ip-allocation-mode for the allocation mode
3. Cordon and drain workloads from the old node pools to the new node pool.
4. Once all workloads are moved and stable, delete the old node pools.

Important constraints from the documentation:

• For Static Block Allocation, migration or updating existing clusters or node pools is not supported; only adding new node pools with a different subnet is supported. This implies that the safe path is to:
  • Add new node pools using the new pod subnet and allocation mode.
  • Move workloads.
  • Remove the old pools.
• Only a single mode of operation can be used per subnet. If a subnet uses Static Block allocation, it cannot use Dynamic IP allocation in a different cluster or node pool with the same subnet and vice versa.

Given these constraints, a safe migration path is limited to the documented pattern of adding new node pools with the new pod subnet and allocation mode, then draining and deleting the old pools. In-place modification of existing pools to add pod subnets or to mix pools with and without pod subnets in the same cluster is not supported.

---

References:

• Configure Azure CNI Pod Subnet - Static Block Allocation and enhanced subnet support in Azure Kubernetes Service (AKS)
• Configure Azure CNI Pod Subnet - Dynamic IP Allocation and enhanced subnet support in Azure Kubernetes Service (AKS)
• Azure Container Networking Interface (CNI) Pod Subnet

Answer 2

Hello @Agustin Cano

How Q&A assist already provided an usefull answer, i hope it fullfield your concern or issue.

One thing tha raised me a doubt was the mentioned version of AKS... AKS is currently testing 1.35 and not GA yet. When you mentioned 1.37 is that a typo?

Best regards.