I'm not able to install the Nvidia and CUDA drivers into my Ubuntu Azure VM.

Kolluru Krishna Mouli Maudga 25 Reputation points
2023-10-09T07:42:57.27+00:00

I want to deploy the Llama2 model using a docker container into my azure VM, to achieve this I need to first create a GPU based VM and install the concerned Nvidia and CUDA drivers into the VM, here is where my problem is, I followed 3 methods to achieve this, {( the VM image I used is Ubuntu 22.04, and the VM family is Standard_NV36adms_A10_v5 )},

  1. I used the Nvidia GPU driver extension to install the drivers which was the recommended approach in the Microsoft website, https://learn.microsoft.com/en-us/azure/virtual-machines/linux/n-series-driver-setup, but it keeps giving provisioning error.User's image
  2. I tried the manual method, the steps for this are also provided in the same link I provided above, the problem I faced here was that the VM wasn't rebooting after I entered "sudo reboot", after this the VM status is always set to "not ready". There was no way I could reconnect to the VM.

3)Then I even tried using the Image provided by Nvidia in the Microsoft Azure Marketplace, to be precise I used this image "NVIDIA GPU-Optimized VMI with vGPU driver - v22.08.0" the issue I faced here is this image has the CUDA driver version 11.2 I need at least 11.8 to run Llama2, I couldn't update the CUDA driver, it wasn't letting me, I also tried other images, "NVIDIA GPU-Optimized VMI - v23.09.1", "NVIDIA GPU-Optimized VMI - v23.03.0" and 2 similar one's the problem with these is when i enter "nvidia-smi", it says no device nvidia found, it should actually provide the specs of the Nvida driver.

Azure Virtual Machines
Azure Virtual Machines
An Azure service that is used to provision Windows and Linux virtual machines.
9,044 questions
0 comments No comments
{count} votes

Accepted answer
  1. deherman-MSFT 38,021 Reputation points Microsoft Employee Moderator
    2023-10-09T20:57:02.12+00:00

    @Kolluru Krishna Mouli Maudga

    The issue installing the NVIDIA GPU Driver Extension for Linux. Might be caused by having Trusted Launch enabled. Try using a VM with Security Type that is set to Standard.
    User's image

    If that does not work for you try following the steps in this thread.

    I launched a generic Ubuntu 22.04 image and was able to successfully install the driver using the extension.

    enter image description here

    Hope this helps! Let me know if you still have issues.


    If you still have questions, please let us know in the "comments" and we would be happy to help you. Comment is the fastest way of notifying the experts.

    If the answer has been helpful, we appreciate hearing from you and would love to help others who may have the same question. Accepting answers helps increase visibility of this question for other members of the Microsoft Q&A community.

    Thank you for helping to improve Microsoft Q&A! User's image

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.