How to run Meta Llama 3.2 on Azure ML Jupyter notebook

Question

I want to use Meta Llama 3.2 model on a directory in my Azure Machine Learning workspace so I can run inference directly within my notebook. However, the only option I am seeing in Azure is to deploy the Meta Llama 3.2 model on a managed compute. I don’t want to deploy it—I want to use the weights directly, without relying on the call API.

Anyone knows how to do this?

Answer 1

@Hadjmbarek Nadia I believe you are referring to deploying the model from Model catalog page of Azure ML studio. From Model catalog page you get an option to deploy the model with one click on managed compute without manually configuring any dependencies. But, if you need to have complete control on the model deployment you will have to first register the same model on the models page and then use it in your notebook or deploy it as an endpoint with your own scoring script. This should be possible if you are able to download the same model from meta page and register it on model's tab of ML studio.

Once you register a model, you can list it and use it in the SDK on notebook or deploy it as an endpoint with your scoring script. I hope this helps!!

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.