Difference between System mode and User mode nodepool

Question

Hi all,

Currently we are using AKS v1.22.6 where we have 2 node pools configured with 1 system mode and 1 user mode. As we understood from the document, System mode is capable of hosting both system pods and user pods(in other words application pods) whereas, User mode is only accepts the user pods/application pods. Please do correct me if I am wrong.

Question here is, on what ground AKS is segregating the system/pods/application pods and make them running on respective node pools(system and user mode).

For instance: Kube-system pods will only run under "system" node pool, and some webserver pods running under "User" node pool.

Thanks in advance.

Answer 1

Hello @Vignesh Murugan ,

Thank you for reaching out to the Microsoft Q&A platform. Happy to answer your question. Your understanding is correct for system mode and user mode nodepools.

Differences discussed here Use system node pools in Azure Kubernetes Service (AKS) - Azure Kubernetes Service | Microsoft Learn

While I don’t have clear explaination on what dictactes scheduling preferences of pods in AKS for the two nodepools. Users can explicitly make system nodepools dedicated for system pods with this –
On System nodepool, AKS automatically assigns the label kubernetes.azure.com/mode: system to its nodes. This causes AKS to prefer scheduling system pods on node pools that contain this label. This label does not prevent you from scheduling application pods on system node pools. However, we recommend you isolate critical system pods from your application pods to prevent misconfigured or rogue application pods from accidentally killing system pods. You can enforce this behavior by creating a dedicated system node pool. Use the CriticalAddonsOnly=true:NoSchedule taint to prevent application pods from being scheduled on system node pools.

You can explicitly deploy a Pod to a specific node pool by setting any of the below:

1. Node Affinity
2. Node selector
3. NodeName.

This forces a Pod to run only on nodes in that node pool. For more example see, Deploying a Pod to a specific node pool

Please "Accept as Answer" and Upvote if the answer provided is useful, so that you can help others in the community looking for remediation for similar issues.