Cannot use nvidia-smi on NV12ads A10 v5 virtual machine

Question

Hello,

I followed the install instructions here: https://learn.microsoft.com/en-us/azure/virtual-machines/linux/n-series-driver-setup#ubuntu

I see the GPU available
azureuser:~$ lspci 0002:00:00.0 3D controller: NVIDIA Corporation GA102GL [A10] (rev a1) 6736:00:02.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function] (rev 80)

Then I get

azureuser:~$ nvidia-smi

NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

Thanks!

Answer 1

Hello @Rafsun Faiz ,

You must disable Secure Boot on the Linux VM after installing the signed kernel module for use with GRID.

The GRID driver redistributed by Azure does not work on many non-NV-series VMs (such as NC, NCv2, NCv3, ND, and NDv2-series VMs), but it does work on NCasT4v3-series.

For more information about specific vGPU and driver branch versions, see the NVIDIA website.

GRID Driver 17.3 currently only supports NCasT4_v3-series VMs. To use this driver, manually download and install GRID Driver 17.3.

For more detailed information please reach out Microsoft official link- https://learn.microsoft.com/en-us/azure/virtual-machines/linux/n-series-driver-setup#nvidia-grid-drivers

Here is the clear document Installing the NVIDIA Driver on an Azure VM- https://forums.developer.nvidia.com/t/installing-nvidia-drivers-cuda-on-azure-nvadsa10-v5-vm-ubuntu-22-04/321128/3#p-1604105-installing-the-nvidia-driver-on-an-azure-vm-1

If you have any further queries, do let us know.

Answer 2

Rafsun Faiz - Your VM detects the GPU at PCI level (lspci), but the NVIDIA kernel driver is not loaded or failed to build, which is why nvidia-smi cannot communicate with the driver.

Most common causes & fixes (A10 / N‑series on Linux):

Driver not installed correctly or missing reboot – NVIDIA drivers require a VM reboot after installation.

Kernel/driver mismatch – the driver was built for a different kernel version than the one currently running.

Secure Boot / vTPM enabled – NVIDIA drivers will not load when Secure Boot is enabled on N‑series VMs.

1. Disable Secure Boot and vTPM on the VM.
2. Install the supported NVIDIA driver for A10 (GRID 17.x+ or supported CUDA driver) using the Microsoft‑documented method.
3. Reboot the VM, then validate with: nvidia-smi

This behavior and the supported installation steps are documented here: Install NVIDIA GPU drivers on N‑series Linux VMs

If nvidia-smi still fails after reboot, it indicates the driver module is not loaded, and the driver must be reinstalled following the above doc exactly.