Azure Kubernetes Service - Why Are System Pods Being Scheduled On User Node Pools?

devopsfj 246 Reputation points
2024-11-13T11:19:56.84+00:00

I have a cluster which is made up of 3 node pools:

system - System Node Pool

project 1 - User Node Pool

project 2 - User Node Pool

On the System node pool, I have the taint set CriticalAddonsOnly:true:NoSchedule.

My issue is that for some reason, system pods, for example CoreDNS, Metric Server are all being schedule on project 1 & project 2 Nodes?

I have not used my capacity on the System Nodes either, there is still loads of space for more Pods.

Why is this happening and how can I stop this? I want system pods on the system node pool only.

Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
2,196 questions
{count} votes

1 answer

Sort by: Most helpful
  1. anashetty 980 Reputation points Microsoft Vendor
    2024-11-18T09:09:47.5566667+00:00

    Hi devopsfj,

    I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to "Accept " the answer.

    Issue:

    Azure Kubernetes Service - Why Are System Pods Being Scheduled On User Node Pools?

    Solution:

    I believe I may have found the issue to what was happening...

    I believe my System Nodes did not have the accepted amount of CPU/Memory left to facilitate all system pods with the CriticalAddonsOnly:true:NoSchedule taint.

    Once I increased my system nodes to have the recommended specs as stated here on MS Docs.

    User's image

    All of the system pods were schedules on the system nodes (apart from Daemon Sets of course)

    If you have any other questions or are still running into more issues, please let me know. Thank you again for your time and patience throughout this issue.

    Please remember to "Accept Answer" if any answer/reply helped, so that others in the community facing similar issues can easily find the solution.

    Thank you.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.