How do I enable GPU VMs for Azure Machine Learning Deep Learning jobs

Grant Fullston 0

Hi all,

I am on a pay as you go subscription service, and I can't seem to request a quota for ANY GPU cluster or node on ANY location. When i follow the click here to view and request quota page, I am only able to request quota extensions for CPU for both clusters and compute instances. I should have enough quota as I have 18 nodes available and some of the GPU clusters only need 6-16 cores. If there is something I am not understanding about how many nodes are used in a core please let me know, and some guidance on how much node space I should request would be lovely. Thank you.

Is there a way to solve this issue without submitting a technical support request ticket. Most resources are pointing me in this direction, I am on a Basic support plan, and would like to try avoid needing to sign up for the $29 per month expert help support plan.

Please help, I need to do some Deep Learning tasks for a large scale business application for financial data.

Screen Shot 2024-03-26 at 3.16.50 pm

Screen Shot 2024-03-26 at 3.13.49 pm

romungi-MSFT 42,116 Reputation points Microsoft Employee

2024-03-26T13:13:44.1166667+00:00

@Grant Fullston If you have some endpoints in use, try to delete them and then re-run your job with the compute to check if the freed nodes will help you complete the job. Guidance on compute quotas is available on the learn page. AFAIK, you should be able to raise a quota increase without having to upgrade your support plan, try to follow the steps on the same page to check if it works.
Grant Fullston 0 Reputation points

2024-03-27T05:54:08.4866667+00:00

@romungi-MSFT Thank you, just learnt the GPU families start with an N, so I am able to request quota limits on these clusters to then get GPU capacity for Deep Learning ML jobs. I did see that these are only available on low priority jobs, I plan to eventually bring my solution into a production environment, which I would hope could have dedicated access to GPU compute.

Would you happen to know about the compute requirements of inference in real time, compared to that of training. As it says in the studio, GPU is needed for training of deep learning ML tasks, do you know if it is also required for inference deployment?
romungi-MSFT 42,116 Reputation points Microsoft Employee

2024-03-28T07:45:27.23+00:00

@Grant Fullston You can use GPU based N series for real time online endpoints. See the list of VMs available. For inference you can use scaling mechanism with smaller VMs to control cost and performance.