Can my web app use a GPU for AI capabilities or will I need to use an Azure VM?

Henrik Vlijter 0

I am running a web app which I deployed through docker. The web app works perfectly besides one important detail, the whisperx ai model I have takes forever to run a transcription (think hours). I run the same ai function on a "T4 GPU" using Google collab and it takes 5 minutes using cuda as the device. This is clearly a problem so I wanted to know if the same functionality exists within Azure. Is this possible?

dupammi 7,955 Reputation points Microsoft Vendor

2024-07-02T06:34:44.37+00:00
Hi @Henrik Vlijter

Thank you for using the Microsoft Q&A platform.

To utilize a GPU for AI in your Azure-deployed web app, you can opt for several approaches: Using Azure Virtual Machines with GPUs (like NC, ND, or NV series) lets you run WhisperX or similar models with CUDA support by provisioning a GPU-enabled VM, setting up CUDA and necessary software, and deploying your app within Docker on this VM. Alternatively, Azure Machine Learning allows you to create a GPU-based compute target, deploy your model, and integrate GPU-accelerated AI into your application through managed AML endpoints. For scalable containerized deployments, Azure Kubernetes Service (AKS) with GPU nodes can be used by creating an AKS cluster with GPU node pools, deploying your app, and configuring it to request GPUs using Kubernetes resources. Azure App Services might offer GPU support in future previews, but the above options currently provide more robust solutions for GPU integration, helping to significantly speed up AI tasks like transcription.

Please refer below resources for more details:

Azure Virtual Machines with GPUs

Azure Machine Learning Documentation

Azure Kubernetes Service Documentation

I hope this helps. Thank you.
Henrik Vlijter 0 Reputation points

2024-07-02T09:53:02.6133333+00:00

Thank you @dupammi I don't have much experience with things like this as a student. Can I please ask a few more questions about whether the app can be accessed through a web url? Also will I need to configure the docker image and gpu when creating the app or is this done another way?
dupammi 7,955 Reputation points Microsoft Vendor

2024-07-02T11:19:28.9266667+00:00

Hi @Henrik Vlijter ,

You can indeed leverage Azure’s GPU accelerated capabilities for AI tasks in your web app by using Azure VMs with GPUs, AKS, or Azure Machine Learning. Each method provides a way to integrate GPU support into your application, making it accessible through a web URL by configuring DNS and ingress appropriately.

Share via

Can my web app use a GPU for AI capabilities or will I need to use an Azure VM?