Share via

Upgrade Azure Kubernetes Cluster from 1.19.7 to 1.20.13

Max Remacle 6 Reputation points
2022-01-11T21:23:56.747+00:00

I am having an issue when trying to upgrade my Kubernetes Cluster hosted on Azure from 1.19.7 to 1.20.13.

Whenever I try to complete the upgrade I get the following error message:
(InsufficientSubnetSize) Pre-allocated IPs 124 exceeds IPs available 123 in Subnet Cidr XX.XXX.X.X/25

I tried to scale down the amount of pods (from 30 per node to 20) per node to free up some extra IP addresses to allow the upgrade to occur, but I couldn't reduce this number. From investigation I see that this number can't be changed after creation(Is that correct?)

Can anyone help provide guidance on what to do in this scenario?

Thanks

Azure Kubernetes Service
Azure Kubernetes Service
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
2,457 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Chris Ekin 6 Reputation points
    2022-07-24T20:16:17.82+00:00

    If it's any use to others, I had the same issue. Reducing the number of nodes in my cluster from 5 to 4 allowed me to do the upgrade. AKS needs to create an additional node temporarily while upgrading, and if there aren't enough IPs available for that it will fail. Reducing the number of nodes gives it space.

    1 person found this answer helpful.
  2. shiva patpi 13,366 Reputation points Microsoft Employee Moderator
    2022-01-12T05:56:10.753+00:00

    Hello @Max Remacle ,
    Correct! Max Pod per node setting can be defined only at the time of new node pool creation - it can't be changed later on.

    Reason for the error is because of this: - https://learn.microsoft.com/en-us/azure/aks/configure-azure-cni#plan-ip-addressing-for-your-cluster

    Basically, you cannot modify a subnet if it has any resources using the subnet and you cannot scale the cluster down if it is in a failed state due to an upgrade and also you cannot change the subnet of an existing cluster.

    But you can change the subnet of a node
    Can move your nodes to another subnet in the same VNET, and maintain connectivity
    You can try to delete a node and not affect the running pods, if there is enough space for the pods on the nodes that remain

    Navigate to the VMSS in the portal then click on Scaling. Then scale the VMSS manually down by at least 1 instance:

    164140-image.png

    This will update the node count in the nodepool and reduce the amount of IP's used and the upgrade can be completed

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.