Is it possible to use CUDA v12 on NVadsA10 v5-series instances in AKS?

Patrick Deubel 25 Reputation points

Hi, I have a "Standard_NV12ads_A10_v5" node pool in my AKS cluster. My application that I deploy crashes because it requires CUDA v12, which in turn is caused by the fact that the image that is used on the node uses CUDA v11.6. I have also used other instances such as the single 80GB A100 which to my suprise uses CUDA v12. Why does the A10 instance use CUDA 11.6, is that because it is a slice of a full GPU? Do I have to use a full GPU or can the CUDA version be updated? I also want to autoscale this node pool so that would have to be automated, is that a road block?

Thanks for any answers!

Azure Virtual Machines
Azure Virtual Machines
An Azure service that is used to provision Windows and Linux virtual machines.
6,292 questions
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
1,647 questions
0 comments No comments
{count} votes

Accepted answer
  1. Prrudram-MSFT 19,956 Reputation points

    Hello @Patrick Deubel

    Thank you for reaching out to the Microsoft Q&A platform.

    The reason why the "Standard_NV12ads_A10_v5" node pool in your AKS cluster is using CUDA v11.6 is because it is based on the NVIDIA Tesla V100 GPU, which supports CUDA 11.6. The A10 GPU is a slice of the V100 GPU and has a smaller number of CUDA cores, which is why it is using the same version of CUDA.

    To use CUDA v12, you will need to use a GPU that supports it, such as the NVIDIA A100 GPU. You can create a new node pool in your AKS cluster that uses the A100 GPU and deploy your application to that node pool. You can also use a custom image that has CUDA v12 installed on it.

    To automate the autoscaling of the node pool, you can use the Kubernetes Horizontal Pod Autoscaler (HPA) to automatically scale the number of pods based on CPU or memory utilization. You can also use the Kubernetes Cluster Autoscaler (CA) to automatically scale the number of nodes in the node pool based on the demand for resources. The HPA and CA can work together to ensure that your application has the resources it needs to run efficiently.

    Please click "Accept as answer" and do a Thumbs-Up if this helps

0 additional answers

Sort by: Most helpful