GPUs are a popular choice for artificial intelligence computations, because they offer parallel processing capabilities and can often execute vision-based inferencing faster than CPUs. To better support artificial intelligence and machine learning applications, AKS Edge Essentials can expose a GPU to the virtual machine's Linux module.
AKS Edge Essentials supports GPU-Paravirtualization (GPU-PV) as the GPU passthrough technology. In other words, the GPU is shared between the Linux virtual machine and the host.
Important
These features include components developed and owned by NVIDIA Corporation or its licensors. By using GPU acceleration features, you are accepting and agreeing to the terms of the NVIDIA End-User License Agreement.
Prerequisites
The GPU acceleration of AKS Edge Essentials currently supports a specific set of GPU hardware. Additionally, use of this feature requires specific versions of Windows.
The supported GPUs and required Windows versions are as follows:
Supported GPUs
GPU passthrough type
Supported Windows versions
NVIDIA GeForce, Quadro, RTX
GPU-PV
Windows 10/11 (Pro, Enterprise, IoT Enterprise)
Important
GPU-PV support might be limited to certain generations of processors or GPU architectures, as determined by the GPU vendor. For more information, see the NVIDIA CUDA for WSL documentation.
Windows 10 users must use the November 2021 update build 19044.1620, or later. After installation, you can verify your build version by running winver at the command prompt.
GPU passthrough is not supported with nested virtualization, such as when you run AKS Edge Essentials in a Windows virtual machine.
System setup and installation
The following sections contain setup and installation information.
For NVIDIA GeForce/Quadro/RTX GPUs, download and install the NVIDIA CUDA-enabled driver for Windows Subsystem for Linux (WSL) to use with your existing CUDA ML workflows. Originally developed for WSL, the CUDA for WSL drivers is also used for AKS Edge Essentials.
Windows 10 users must also install WSL because some of the libraries are shared between WSL and AKS Edge Essentials.
Install or upgrade AKS Edge Essentials to the May 2024 release, or later. For more information, see Update your AKS Edge Essentials clusters. The GPU-PV is supported on both k8s and k3s Kubernetes distributions.
Enable GPU-PV in your AKS Edge Essentials deployment
Step 1: single machine configuration parameters
You can generate the parameters you need to create a single machine cluster and add the necessary GPU-PV configuration parameters using the following commands.
This script only focuses on the GPU-PV configuration; you should also make other necessary general updates according to your own AKS Edge Essentials deployment:
User.AcceptGpuWarning: Set this parameter to true to automatically accept the confirmation message when you enable GPU-PV on AKS Edge Essentials.
LinuxNode.GpuPassthrough.Name: Describes the GPU model that's deployed in this machine, with proper drivers installed.
LinuxNode.GpuPassthrough.Type: Describes the GPU passthrough type. Only ParaVirtualization is currently supported.
LinuxNode.GpuPassthrough.Count: Describes how many GPUs are installed on this machine.
Step 2: create a single machine cluster
You can now run the New-AksEdgeDeployment cmdlet to deploy a single-machine AKS Edge cluster with a single Linux control plane node. You can use the JSON object generated in step 1 and pass it as a string:
Verify that the plugin is running and the NVIDIA GPU is detected by checking the logs of the deviceplugin pod using the kubectl get pods -A and kubectl logs $podname -n kube-system commands:
Get started with a sample workload
Prepare a workload YAML file named gpu-workload.yaml: