Edit

What is NVIDIA CUDA?

NVIDIA CUDA is NVIDIA's parallel computing platform and programming model for GPU computing. It provides an end-to-end stack of drivers, compilers, runtimes, and libraries that let you run AI, high-performance computing (HPC), and other accelerated workloads on NVIDIA GPUs. CUDA gives you a programming model, math and communication libraries, and tooling that popular frameworks such as PyTorch, TensorFlow, JAX, and vLLM build on. For AMD GPUs, the comparable platform is AMD ROCm.

On Azure, CUDA is the software layer you install and use to unlock the compute capabilities of NVIDIA GPU-powered virtual machines (VMs). CUDA runs on both Linux and Windows.

Key components of the CUDA platform

CUDA is a full stack rather than a single driver. Its main building blocks include:

  • CUDA Toolkit (compiler and runtime): The nvcc compiler, the CUDA runtime, and the CUDA C++ programming model that let you write and build GPU-accelerated code.
  • Math and compute libraries: GPU-accelerated libraries such as cuBLAS, cuFFT, cuSPARSE, cuDNN (deep learning primitives), and CUTLASS that frameworks call for performance-critical operations.
  • NCCL (NVIDIA Collective Communications Library): A collective-communications library used to scale training and HPC workloads across many GPUs.
  • Compilers and tools: An LLVM-based compiler, plus profiling and debugging tools such as Nsight Systems, Nsight Compute, and cuda-gdb.

A compute-only stack

CUDA targets the general-purpose compute capabilities of a GPU (the parallel math that drives AI, machine learning, and HPC workloads) and doesn't use the graphics rasterization pipeline that turns 3D geometry into rendered pixels. In other words, CUDA treats the GPU as a massively parallel math engine, not as a display or rendering device.

This distinction matters when you choose a VM size. Azure offers both compute-only NVIDIA GPU sizes, such as the ND-H100-v5 series, and graphics-capable NVIDIA GPU sizes, such as the NVadsA10 v5 series, that include a rasterization pipeline for professional visualization and virtual desktop scenarios. CUDA runs on both categories, because it uses the GPU's compute units regardless of whether the hardware also supports rasterization.

Azure GPU VM sizes with CUDA support

The following Azure GPU VM sizes use NVIDIA GPUs and support CUDA. Use the links to review full specifications for each series.

VM size series NVIDIA GPU Typical workloads
ND-GB200-v6 series NVIDIA GB200 NVL (Grace Blackwell) Frontier-scale generative AI training and inference.
ND-H200-v5 series NVIDIA H200 Large-scale deep learning training and inference, generative AI, and HPC.
ND-H100-v5 series NVIDIA H100 Large-scale deep learning training and inference, and tightly coupled HPC.
NCads H100 v5 series NVIDIA H100 NVL Mid-scale training, inference, and HPC.
NC A100 v4 series NVIDIA A100 Training, inference, and HPC.
NCasT4 v3 series NVIDIA T4 Inference, small-scale training, and visualization.
NVadsA10 v5 series NVIDIA A10 GPU-accelerated graphics, virtual desktops, and light inference.

For the full list of GPU-accelerated sizes, see the GPU-accelerated VM sizes overview.

Supported operating systems

CUDA supports both Linux and Windows. The specific distributions, kernel versions, and driver versions that NVIDIA validates depend on the CUDA Toolkit release and the GPU:

  • Linux: CUDA supports a broad set of enterprise distributions, including Ubuntu, Red Hat Enterprise Linux (RHEL), Rocky Linux, SUSE Linux Enterprise Server (SLES), and Debian.
  • Windows: CUDA supports Windows Server and Windows 11. You can also run CUDA workloads in Windows Subsystem for Linux (WSL 2).

Because NVIDIA updates the validated OS, driver, and CUDA Toolkit version combinations with each release, always confirm the current requirements in the NVIDIA CUDA Toolkit documentation before you deploy.

Install CUDA on Azure NVIDIA GPU VMs

The recommended way to get a preconfigured environment on NVIDIA GPU VMs is to deploy an Azure Marketplace image that already includes the NVIDIA GPU driver and CUDA. You can also install the NVIDIA GPU driver and CUDA Toolkit manually on a supported distribution.

For Azure-specific, step-by-step guidance, see:

For NVIDIA's own installation and reference documentation, see:

Related content

  • Last updated on