Share via

Is there a parameter for setting max size for batch scale up of nodes in AKS?

vedant.lodha 0 Reputation points
2024-02-06T04:26:44.4466667+00:00

Hi! I have multiple clusters in an azure subscription with a CPU quota set. Now one of my cluster running some batch application is trying to scale up the nodes. While the new scale up request is well within the autoscaler limits, it exceeds the CPU quota, hence failing the autoscale operation. EG: Azure CPU quota: 100 CPUs. Used quota: 60 CPUs Node type: 8 cpu instance Now the cluster autoscaler makes a request for 6 nodes scale up(within the autoscale limits) however since the quota is hit(108 CPUs), the autoscale fails. I would like to know if i can set a max batch scale up size here so that rest of the pods can stay in pending state while the cluster scales up atleast 5 nodes.

Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
1,999 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Sam Cogan 10,347 Reputation points MVP
    2024-02-06T08:37:14.5+00:00

    I would recommend raising a quota support request to increase the CPU quota to a number that matches your requirement. This can be done quickly and is usually approved within minutes.,

  2. Anveshreddy Nimmala 3,535 Reputation points Microsoft Vendor
    2024-02-07T07:03:55.04+00:00

    Hello @vedant.lodha, Thankyou for replying back.

    you can set the maxNodesPerScaleUp parameter in the cluster autoscaler configuration to limit the maximum number of nodes that can be added to the cluster in a single scale-up event.

    This parameter specifies the maximum number of nodes that can be added to the cluster in a single scale-up event. az aks update --resource-group myResourceGroup --name myAKSCluster --enable-cluster-autoscaler --min-count 1 --max-count 25 --nodepool-name myNodePool --max-nodes-per-scale-up 3 ex: if you want to limit the maximum number of nodes that can be added to the cluster to 3 in a single scale-up event, you can set the maxNodesPerScaleUp parameter to 3 in the cluster autoscaler configuration. This will ensure that the autoscaler creates scale-up requests with a smaller batch size Please note that setting a smaller batch size may result in longer wait times for your applications to start running. Therefore, it is recommended to set the batch size based on your application requirements Hope this answer helps you , please consider accepting the answer to help increase visibility of this question for other members of the Microsoft Q&A community. If not, please let us know what is still needed in the comments so the question can be answered. Thank you for helping to improve Microsoft Q&A!.