This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(80, 54%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAG%3C/text%3E%3C/svg%3E)
I am trying to find the best way to run my machine learning models on GPUs for inference as an http request. Do Azure functions support GPUs? if not, what are other options I can look into?
note: I also want to use packaged models, not necessarily ones of my own creation (such as easyOCR for python)
Hi,
If you need GPU support on ML inference the only supported option is the Azure Kubernetes Service as stated in this documentation
For guidance on deploying an ML model to AKS, please refer to this documenation on deploying to AKS
Thanks!
The link you provided contains a note that mentions that azure machine learning endpoints "provide an improved, simpler deployment experience. Endpoints support both real-time and batch inference scenarios"
Should I then explore this option instead of AKS?
I also found this Article that says that it supports GPUs. Is that the case?
The managed endpoint are currently in Preview and documetation indeed indicates that GPU's will be supported. However the documentation states that this option is not recommended for production workloads.
So IMHO the managed endpoint option is excellent for research, development and testing use-cases. But for production workloads I would recommend using AKS.
Right I understand!
One more thing, I also found this repository that describes a way of deploying GPU enabled Azure functions using AKS
Is this a viable option?
The link you provide seems like a good viable option. I think it's certainly worth a try.