AKS Node Pool

Question

Hello,

We want to use AKS as shared resource for multiple application. Each application will have a separate namespace and permission limited to its namespace.

The question is how should we setup the node pool to deploy our applications ? Is it ok to use the default agent node pool with min node count set as '2' and auto scale enabled ? I mean can we go ahead and use the default node pool setup to deploy the applications separated by namespaces and increase the max node count depends on usage.

Answer 1

You can have a user-node pool for sure. But make sure that, you create it with special label

az aks nodepool add \  
    --resource-group rg \  
    --cluster-name aks \  
    --name nodepool2 \  
    --node-count 2 \  
    --node-vm-size Standard_DS2_v2 \  
    --labels "vmsize=small"  

Here the label is vmsize=small . Through this label, we’ll be able to specify which nodes the pods should land on.
When we created the node pools, specify --labels option. An example below:

kind: Deployment  
apiVersion: apps/v1  
metadata:  
  name: app1  
spec:  
  replicas: 16  
  selector:  
    matchLabels:  
      app: app1  
  template:  
    metadata:  
      labels:  
        app: app1  
    spec:  
      containers:  
        - name: app1  
          image: debian:latest  
          command: ["/bin/bash"]  
          args: ["-c", "hello world; sleep 30; done"]  
      nodeSelector:  
        vmsize: small  

Answer 2

Hi @Niren Adhikary (NAD)

Thanks for reaching Q & A forum.

To ensure your cluster operates reliably, you should run at least 2 (two) nodes in the default node pool, as essential system services are running across this node pool.

System node pools serve the primary purpose of hosting critical system pods such as CoreDNS and tunnelfront. User node pools serve the primary purpose of hosting your application pods.

-----

If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

Answer 3

AKS clusters can be shared across multiple tenants in different scenarios and ways. In some cases, diverse applications can run in the same cluster. In other cases, multiple instances of the same application can run in the same shared cluster, one for each tenant. All these types of sharing are frequently described using the umbrella term multitenancy.

For example, the following picture shows the typical SaaS provider model that hosts multiple instances of the same application on the same cluster, one for each tenant. Each application lives in a separate namespace.

In a fully multitenant deployment, a single application serves the requests of all the tenants, and all the Azure resources are shared, including the AKS cluster. In this context, you only have one set of infrastructure to deploy, monitor, and maintain. All the tenants use the resource, as illustrated in the following diagram:

To keep up with the traffic demand that's generated by tenant applications, you can enable the cluster autoscaler to scale the agent nodes of your Azure Kubernetes Service (AKS). Autoscaling helps systems remain responsive in the circumstances where autoscaling is needed. When you enable autoscaling for a node pool, you specify a minimum and a maximum number of nodes based on the expected workload sizes. By configuring a maximum number of nodes, you can ensure enough space for all the tenant pods in the cluster, regardless of the namespace they run in.

When the traffic increases, cluster autoscaling adds new agent nodes to avoid pods going into a pending state, due to a shortage of resources in terms of CPU and memory.

Likewise, when the load diminishes, cluster autoscaling decreases the number of agent nodes in a node pool, based on the specified boundaries, which helps reduce your operational costs.

To reduce the risk of downtimes that may affect tenant applications during cluster or node pool upgrades, schedule AKS Planned Maintenance to occur during off-peak hours. Planned Maintenance allows you to schedule weekly maintenance windows to update the control plane of the AKS clusters that run tenant applications and node pools, which minimizing workload impact. You can schedule one or more weekly maintenance windows on your cluster by specifying a day or time range on a specific day. All maintenance operations will occur during the scheduled windows.

When you share an AKS cluster between multiple teams within an organization, you need to implement the principle of least privilege to isolate different tenants from one another. In particular, you need to make sure that users have access only to their Kubernetes namespaces and resources when using tools, such as kubectl, Helm, Flux, Argo CD, or other types of tools.

----------

--please don't forget to upvote and Accept as answer if the reply is helpful--

Answer 4

Hi @Manu Philip

Thanks for the explanation. But it does not answer my query which I mentioned in the last comment. My question is specific to Adding User Node pool in addition to default system node pool