Manage node pools for a cluster (AKS on Azure Stack HCI 22H2)

Applies to: AKS on Azure Stack HCI 22H2, AKS on Windows Server

Note

For information about managing node pools in AKS on Azure Stack HCI 23H2, see Manage node pools.

In AKS enabled by Azure Arc, nodes of the same configuration are grouped together into node pools. These node pools contain the underlying VMs that run your applications. This article shows you how to create and manage node pools for a cluster in AKS Arc.

Note

This feature enables greater control over how to create and manage multiple node pools. As a result, separate commands are required for create, update, and delete operations. Previously, cluster operations through New-AksHciCluster or Set-AksHciCluster were the only option to create or scale a cluster with one Windows node pool and one Linux node pool. This feature exposes a separate operation set for node pools that require the use of the node pool commands New-AksHciNodePool, Set-AksHciNodePool, Get-AksHciNodePool, and Remove-AksHciNodePool to execute operations on an individual node pool.

Before you begin

We recommend that you install version 1.1.6. If you already have the PowerShell module installed, run the following command to find the version:

Get-Command -Module AksHci

If you need to update PowerShell, following the instructions in Upgrade the AKS host.

Create an AKS cluster

To get started, create an AKS cluster with a single node pool. The following example uses the New-AksHciCluster command to create a new Kubernetes cluster with one Linux node pool named linuxnodepool, which has 1 node. If you already have a cluster deployed with an older version of AKS, and you wish to continue using your old deployment, you can skip this step. You can still use the new set of node pool commands to add more node pool to your existing cluster.

New-AksHciCluster -name mycluster -nodePoolName linuxnodepool -nodeCount 1 -osType linux

Note

The old parameter set for New-AksHciCluster is still supported.

Add a node pool

The cluster named mycluster*`, created in the previous step, has a single node pool. You can add a second node pool to the existing cluster using the New-AksHciNodePool command. The following example creates a Windows node pool named windowsnodepool with one node. Make sure that the name of the node pool is not the same name as any existing node pool.

New-AksHciNodePool -clusterName mycluster -name windowsnodepool -count 1 -osType Windows -osSku Windows2022

Get configuration information of a node pool

To see the configuration information of your node pools, use the Get-AksHciNodePool command.

Get-AksHciNodePool -clusterName mycluster

Example output:

ClusterName  : mycluster
NodePoolName : linuxnodepool
Version      : v1.20.7
OsType       : Linux
NodeCount    : 1
VmSize       : Standard_K8S3_v1
Phase        : Deployed

ClusterName  : mycluster
NodePoolName : windowsnodepool
Version      : v1.20.7
OsType       : Windows
NodeCount    : 1
VmSize       : Standard_K8S3_v1
Phase        : Deployed

To see the configuration information of one specific node pool, use the -name parameter in Get-AksHciNodePool.

Get-AksHciNodePool -clusterName mycluster -name linuxnodepool

Example output:

ClusterName  : mycluster
NodePoolName : linuxnodepool
Version      : v1.20.7
OsType       : Linux
NodeCount    : 1
VmSize       : Standard_K8S3_v1
Phase        : Deployed
Get-AksHciNodePool -clusterName mycluster -name windowsnodepool

Example output:

ClusterName  : mycluster
NodePoolName : windowsnodepool
Version      : v1.20.7
OsType       : Windows
NodeCount    : 1
VmSize       : Standard_K8S3_v1
Phase        : Deployed

Note

If you use the new parameter sets in New-AksHciCluster to deploy a cluster and then run Get-AksHciCluster to get the cluster information, the fields WindowsNodeCount and LinuxNodeCount in the output will return 0. To get the accurate number of nodes in each node pool, use the command Get-AksHciNodePool with the specified cluster name.

Scale a node pool

You can scale the number of nodes up or down in a node pool.

To scale the number of nodes in a node pool, use the Set-AksHciNodePool command. The following example scales the number of nodes to 3 in a node pool named linuxnodepool in the mycluster cluster.

Set-AksHciNodePool -clusterName mycluster -name linuxnodepool -count 3

Scale control plane nodes

Management of control plane nodes hasn't changed. The way in which they are created, scaled, and removed remains the same. You will still deploy control plane nodes with the New-AksHciCluster command with the parameters controlPlaneNodeCount and controlPlaneVmSize with the default values of 1 and Standard_A4_V2, respectively, if you don't provide any values.

You may need to scale the control plane nodes as the workload demand of your applications change. To scale the control plane nodes, use the Set-AksHciCluster command. The following example scales the control plane nodes to 3 in mycluster cluster, which was created in the previous steps.

Set-AksHciCluster -name mycluster -controlPlaneNodeCount 3

Delete a node pool

If you need to delete a node pool, use the Remove-AksHciNodePool command. The following example removes the node pool named windowsnodepool from the mycluster cluster.

Remove-AksHciNodePool -clusterName mycluster -name windowsnodepool

Specify a taint for a node pool

When creating a node pool, you can add taints to that node pool. When you add a taint, all nodes within that node pool also get that taint. For more information about taints and tolerations, see Kubernetes Taints and Tolerations.

Setting node pool taints

To create a node pool with a taint, use New-AksHciNodePool. Specify the name taintnp, and use the -taints parameter to specify sku=gpu:noSchedule for the taint.

New-AksHciNodePool -clusterName mycluster -name taintnp -count 1 -osType linux -taints sku=gpu:NoSchedule

Note

A taint can only be set for node pools during node pool creation.

Run the following command to make sure the node pool was successfully deployed with the specified taint.

Get-AksHciNodePool -clusterName mycluster -name taintnp

Output

Status       : {Phase, Details}
ClusterName  : mycluster
NodePoolName : taintnp
Version      : v1.20.7-kvapkg.1
OsType       : Linux
NodeCount    : 1
VmSize       : Standard_K8S3_v1
Phase        : Deployed
Taints       : {sku=gpu:NoSchedule}

In the previous step, you applied the sku=gpu:NoSchedule taint when you created your node pool. The following basic example YAML manifest uses a toleration to allow the Kubernetes scheduler to run an NGINX pod on a node in that node pool.

Create a file named nginx-toleration.yaml, and copy the information in the following text.

apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  containers:
  - image: mcr.microsoft.com/oss/nginx/nginx:1.15.9-alpine
    name: mypod
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 1
        memory: 2G
  tolerations:
  - key: "sku"
    operator: "Equal"
    value: "gpu"
    effect: "NoSchedule"

Then, schedule the pod using the following command.

kubectl apply -f nginx-toleration.yaml

To verify that the pod was deployed, run the following command:

kubectl describe pod mypod
[...]
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
                 sku=gpu:NoSchedule
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  32s   default-scheduler  Successfully assigned default/mypod to moc-lk4iodl7h2y
  Normal  Pulling    30s   kubelet            Pulling image "mcr.microsoft.com/oss/nginx/nginx:1.15.9-alpine"
  Normal  Pulled     26s   kubelet            Successfully pulled image "mcr.microsoft.com/oss/nginx/nginx:1.15.9-alpine" in 4.529046457s
  Normal  Created    26s   kubelet            Created container mypod

Next steps