While upgrading the AKS cluster version, we are getting the error as: Failed to save Kubernetes service 'aks-crushsftp-prd'. Error: Pre-allocated IPs 333 exceeds IPs available 246 in Subnet Cidr 10.56.27.0/24, Subnet Name ciosftp-prd-eastu2-aks.

Krishna satish Polasi Venkata 20 Reputation points
2023-03-25T08:02:17.5733333+00:00

While upgrading the AKS cluster version, we are getting the error as: Failed to save Kubernetes service 'aks-crushsftp-prd'. Error: Pre-allocated IPs 333 exceeds IPs available 246 in Subnet Cidr 10.56.27.0/24, Subnet Name ciosftp-prd-eastu2-aks.

User's image

Azure Virtual Machines
Azure Virtual Machines
An Azure service that is used to provision Windows and Linux virtual machines.
7,158 questions
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
1,867 questions
Microsoft Entra ID
Microsoft Entra ID
A Microsoft Entra identity service that provides identity management and access control capabilities. Replaces Azure Active Directory.
19,562 questions

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Andrei Barbu 2,576 Reputation points Microsoft Employee
    2023-03-25T08:17:51.7666667+00:00

    Hello Krishna satish Polasi Venkata

    During the upgrade operation, buffer node(s) are added in order to run the workload of the node(s) that are being upgraded. After the upgrade is finished, the buffer node is removed. You can read more here: https://learn.microsoft.com/en-us/azure/aks/upgrade-cluster?tabs=azure-cli#upgrade-an-aks-cluster

    The number of buffer node(s) are determined based on the node surge you specified. At this moment, if you don't specify the surge value, it adds an extra node by default. This behavior will change in Kubernetes version 1.27 when the surge value will be 10%.

    That node(s) will require IP addresses and the number of IP addresses depends on the maxPods parameter value you use.

    That message is saying that you don't have enough IP addresses for the buffer node(s) needed during the upgrade operation.

    In this situation, you can reduce the cluster nodes by scaling down to free-up IP addresses for the upgrade.

    If scaling down isn't an option, and your virtual network CIDR has enough IP addresses, try to add a node pool that has a unique subnet:

    Add a new user node pool in the virtual network on a larger subnet.

    Switch the original node pool to a system node pool type.

    Scale up the user node pool.

    Scale down the original node pool.

    This official documentation is intended to address the situation you are facing, so I would highly recommend you to follow it: https://learn.microsoft.com/en-us/troubleshoot/azure/azure-kubernetes/error-code-subnetisfull-upgrade

    I hope this is helpful. If any clarification needed, let me know and I will do my best to answer.

    Please "Accept as Answer" and Upvote if it helped, so that it can help others in the community looking for help on similar topics.

    Thank you!

    0 comments
  2. Erkan Sahin 830 Reputation points
    2023-03-25T11:29:44.96+00:00

    The error message suggests that there are more pre-allocated IP addresses for the Kubernetes service 'aks-crushsftp-prd' than the number of available IP addresses in the subnet CIDR 10.56.27.0/24.

    To resolve this issue, you can try one of the following solutions:

    1. Increase the size of the subnet CIDR: You can increase the size of the subnet CIDR to accommodate more pre-allocated IP addresses for the Kubernetes service.
    2. Decrease the number of pre-allocated IP addresses: You can decrease the number of pre-allocated IP addresses for the Kubernetes service 'aks-crushsftp-prd' to fit within the available IP addresses in the subnet CIDR 10.56.27.0/24.
    3. Create a new subnet with a larger CIDR: You can create a new subnet with a larger CIDR and move the Kubernetes service 'aks-crushsftp-prd' to the new subnet.

    Before making any changes, it is recommended to review your network topology and ensure that the changes will not affect any other resources that are using the subnet.

    0 comments