Share via

GPU acceleration for Azure IoT Edge for Linux on Windows

  • Article

Applies to: yes icon IoT Edge 1.1

Important

IoT Edge 1.1 end of support date was December 13, 2022. Check the Microsoft Product Lifecycle for information about how this product, service, technology, or API is supported. For more information about updating to the latest version of IoT Edge, see Update IoT Edge.

GPUs are a popular choice for artificial intelligence computations, because they offer parallel processing capabilities and can often execute vision-based inferencing faster than CPUs. To better support artificial intelligence and machine learning applications, Azure IoT Edge for Linux on Windows (EFLOW) can expose a GPU to the virtual machine's Linux module.

Azure IoT Edge for Linux on Windows supports several GPU passthrough technologies, including:

  • Direct Device Assignment (DDA) - GPU cores are allocated either to the Linux virtual machine or the host.

  • GPU-Paravirtualization (GPU-PV) - The GPU is shared between the Linux virtual machine and the host.

You must select the appropriate passthrough method during deployment to match the supported capabilities of your device's GPU hardware.

Important

These features may include components developed and owned by NVIDIA Corporation or its licensors. The use of the components is governed by the NVIDIA End-User License Agreement located on NVIDIA's website.

By using GPU acceleration features, you are accepting and agreeing to the terms of the NVIDIA End-User License Agreement.

Prerequisites

The GPU acceleration features of Azure IoT Edge for Linux on Windows currently supports a select set of GPU hardware. Additionally, use of this feature may require specific versions of Windows.

The supported GPUs and required Windows versions are listed below:

Supported GPUs GPU Passthrough Type Supported Windows Versions
NVIDIA T4, A2 DDA Windows Server 2019
Windows Server 2022
Windows 10/11 (Pro, Enterprise, IoT Enterprise)
NVIDIA GeForce, Quadro, RTX GPU-PV Windows 10/11 (Pro, Enterprise, IoT Enterprise)
Intel iGPU GPU-PV Windows 10/11 (Pro, Enterprise, IoT Enterprise)

Important

GPU-PV support may be limited to certain generations of processors or GPU architectures as determined by the GPU vendor. For more information, see Intel's iGPU driver documentation or NVIDIA's CUDA for WSL Documentation.

Windows Server 2019 users must use minimum build 17763 with all current cumulative updates installed.

Windows 10 users must use the November 2021 update build 19044.1620 or higher. After installation, you can verify your build version by running winver at the command prompt.

GPU passthrough is not supported with nested virtualization, such as running EFLOW in a Windows virtual machine.

System setup and installation

The following sections contain setup and installation information, according to your GPU.

NVIDIA T4/A2 GPUs

For T4/A2 GPUs, Microsoft recommends installing a device mitigation driver from your GPU's vendor. While optional, installing a mitigation driver may improve the security of your deployment. For more information, see Deploy graphics devices using direct device assignment.

Warning

Enabling hardware device passthrough may increase security risks. Microsoft recommends a device mitigation driver from your GPU's vendor, when applicable. For more information, see Deploy graphics devices using discrete device assignment.

NVIDIA GeForce/Quadro/RTX GPUs

For NVIDIA GeForce/Quadro/RTX GPUs, download and install the NVIDIA CUDA-enabled driver for Windows Subsystem for Linux (WSL) to use with your existing CUDA ML workflows. Originally developed for WSL, the CUDA for WSL drivers are also used for Azure IoT Edge for Linux on Windows.

Windows 10 users must also install WSL because some of the libraries are shared between WSL and Azure IoT Edge for Linux on Windows.

Intel iGPUs

For Intel iGPUs, download and install the Intel Graphics Driver with WSL GPU support.

Windows 10 users must also install WSL because some of the libraries are shared between WSL and Azure IoT Edge for Linux on Windows.

Enable GPU acceleration in your Azure IoT Edge Linux on Windows deployment

Once system setup is complete, you are ready to create your deployment of Azure IoT Edge for Linux on Windows. During this process, you must enable GPU as part of EFLOW deployment.

For example, the command below creates a virtual machine with an NVIDIA A2 GPU assigned.

Deploy-Eflow -gpuPassthroughType "DirectDeviceAssignment" -gpuCount 1 -gpuName "NVIDIA A2"

Once installation is complete, you are ready to deploy and run GPU-accelerated Linux modules through Azure IoT Edge for Linux on Windows.

Next steps