When starting the AKS cluster one of the node has high memory consumption
Hi we have AKS cluster with 4 nodes. And one of the node is tainted to schedule specific pods. we are starting and stopping our AKS cluster daily because of budget constraint. We observed that when we start our AKS cluster one of the node is consuming high memory say between 95% to 103% by the way we are using Standard D4s v3 (4 vcpus, 16 GiB memory) for each of the node. The reason why there is high memory utilization is because most of the pods are trying to get scheduled on that node rather than other two nodes. Note: we cannot deploy any other pods on the 4th node because it is tained. we have around 126 pods including the system pods. And we have around 9 pods in the tained node including system pods so this 9 pods will be fixed and will be scheduled on the 4th node. So the rest 117 pods which includes system pods should be scheduled on the remaining three nodes. So is there a way when we start the cluster the 117 pods can be distributed evenly among the other 3 nodes?
So that one particular node doesn't face the memory pressure.
@SRIJIT-BOSE-MSFT any idea on this?