Edit

Create a fully managed GPU node pool on Azure Kubernetes Service (AKS)

Running NVIDIA GPU workloads on Azure Kubernetes Service (AKS) traditionally requires you to install and maintain the NVIDIA GPU driver, Kubernetes device plugin, and a GPU metrics exporter on each GPU node. These components enable GPU scheduling, container-level GPU access, and telemetry, but installing them manually or through the NVIDIA GPU Operator adds operational overhead.

By using fully managed GPU nodes, AKS installs and maintains the NVIDIA GPU driver, device plugin, and Data Center GPU Manager (DCGM) metrics exporter for you. GPU node pool creation becomes a single step, and GPU capacity behaves like any other AKS node pool. You can use managed GPU nodes with manually created GPU node pools and with node auto-provisioning (NAP), including AKS Automatic clusters.

You configure a managed GPU node pool through two fields under gpuProfile.nvidia:

  • managementMode (Managed or Unmanaged) controls whether AKS installs the full managed GPU stack (driver, device plugin, and DCGM metrics exporter) or the driver only. The default is Unmanaged.
  • migStrategy (None, Single, or Mixed) sets the Multi-Instance GPU (MIG) strategy for supported GPU SKUs such as A100 and H100. The default is None.

In this article, you provision a managed GPU node pool, optionally enable MIG, verify the stack, and run a sample GPU workload.

Before you begin

  • This article assumes you have an existing AKS cluster. If you don't have a cluster, create one using the Azure CLI, Azure PowerShell, or the Azure portal.
  • You need the Azure CLI version 2.85.0 or later installed. To find the version, run az --version. If you need to install or upgrade, see Install Azure CLI.
  • Get the credentials for your AKS cluster with az aks get-credentials before running the kubectl examples in this article.

Managed GPU components

A managed GPU node pool can include the following components on every node:

Component What it does What AKS manages
NVIDIA GPU driver Kernel modules and user-space libraries that let the OS and containers talk to the GPU hardware. Driver version selection, installation at node provisioning, and reinstallation after node image upgrades.
NVIDIA Kubernetes device plugin DaemonSet-equivalent that advertises GPU resources (nvidia.com/gpu, nvidia.com/mig-*) to the kubelet so pods can request them. Deployment, configuration (including MIG strategy), and lifecycle on each GPU node.
NVIDIA DCGM and DCGM metrics exporter Data Center GPU Manager collects GPU health and utilization data and exposes Prometheus metrics (for example, DCGM_FI_DEV_GPU_UTIL, DCGM_FI_DEV_GPU_TEMP) on port 19400. Installation, service enablement, and the kubernetes.azure.com/dcgm-exporter=enabled node label used to scrape metrics.
GPU health signals NPD signals that surface GPU-specific node conditions such as UnhealthyNvidiaDevicePlugin and UnhealthyNvidiaDCGMServices. NPD monitoring and condition reporting on GPU nodes.

Install profiles

Two gpuProfile fields decide which of those components AKS installs:

  • gpuProfile.driver (Install or None): whether AKS installs the NVIDIA GPU driver.
  • gpuProfile.nvidia.managementMode (Managed or Unmanaged): whether AKS also installs the Kubernetes-facing GPU stack on top of the driver.

Together, they produce three install profiles:

Install profile CLI flags What AKS installs and manages
Full managed stack --enable-managed-gpu=true (or neither flag) All four components above: driver, device plugin, DCGM metrics exporter, and GPU health monitoring in NPD.
Driver only (default) --enable-managed-gpu=false NVIDIA GPU driver only. You install and manage the device plugin, metrics exporter, and health monitoring yourself (for example, with the NVIDIA GPU Operator).
None (BYO) --enable-managed-gpu=false --gpu-driver None Nothing. AKS doesn't install any of the four components. You own the full stack. See Bring your own GPU driver.

Defaults and overrides

  • Defaults: If you don't pass --enable-managed-gpu or --gpu-driver, AKS applies the Driver only profile on the node pool created with an NVIDIA GPU-enabled VM size.
  • Override: managementMode: Managed requires the driver, so --gpu-driver None is ignored when --enable-managed-gpu=true and the driver is still installed. To skip the driver, set both --enable-managed-gpu=false and --gpu-driver None.
  • Immutability: managementMode, migStrategy, and driver are all fixed at creation time. To change profile, create a new node pool.

Limitations

  • This feature currently supports NVIDIA GPU-enabled virtual machine (VM) sizes only.
  • Updating a general-purpose node pool to add a GPU VM size isn't supported on AKS.
  • Windows node pools aren't supported with this feature, because GPU metrics aren't supported. When you create Windows GPU node pools, AKS automatically installs and manages the drivers and DirectX device plugin. For more information, see the AKS Windows GPU documentation.
  • Migrating your existing multi-instance GPU node pools to use this feature isn't supported.
  • In-place upgrades from an existing NVIDIA GPU node pool to a managed GPU node pool aren't supported. To migrate, cordon and drain your existing GPU nodes, then redeploy your workloads to a new GPU node pool created with --enable-managed-gpu=true. For more information, see Resize node pools on AKS.
  • The managementMode, migStrategy, and driver fields under gpuProfile are immutable after node pool creation. To change these values, create a new node pool.

Note

GPU-enabled VMs contain specialized hardware subject to higher pricing and region availability. For more information, see the pricing tool and region availability.

Create an AKS-managed GPU node pool

Add a managed GPU node pool to an existing AKS cluster by passing --enable-managed-gpu=true to az aks nodepool add. AKS sets gpuProfile.nvidia.managementMode to Managed and installs the GPU driver, device plugin, and DCGM metrics exporter automatically.

For AKS Automatic or AKS Standard clusters that use NAP, configure the managed GPU stack on the AKSNodeClass instead of using az aks nodepool add.

To use the default Ubuntu operating system (OS) SKU, you create the node pool without specifying an OS SKU. The node pool is configured for the default operating system based on the Kubernetes version of the cluster.

  1. Add a node pool to your cluster using the az aks nodepool add command with the --enable-managed-gpu=true flag.

    az aks nodepool add \
    --resource-group myResourceGroup \
    --cluster-name myAKSCluster \
    --name gpunp \
    --node-count 1 \
    --node-vm-size Standard_NC4as_T4_v3 \
    --node-taints sku=gpu:NoSchedule \
    --enable-managed-gpu=true

  2. Confirm that the managed NVIDIA GPU software components are installed successfully:

    az aks nodepool show \
    --resource-group myResourceGroup \
    --cluster-name myAKSCluster \
    --name gpunp

    Your output should include the following values:

    ...
"gpuProfile": {
    "driver": "Install",
    "driverType": "",
    "nvidia": {
        "managementMode": "Managed",
        "migStrategy": null
    }
},
...
  • Last updated on