Azure AI Foundry - Deployment of custom Meta Llama finetuned model fails

Question

Hi, I have successfully finetuned a Meta Llama 3.1-8B-Instruct model in serverless mode using synthetic data generated by gpt4. Both the foundry hub and project that were used for finetuning, are created at the westus3 region where the particular Meta model seems to be available for finetuning and deployment according to this https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/deploy-models-serverless-availability. But when I try to deploy it using the serverless API (there is no other option), it constantly fails. And I am unable to find any deployment logs for starters. Any ideas or hints would be much appreciated, thanks in advance.

Answer 1

Hi @Dimitris Vasiliou,

The issue might be related to Azure role-based access control (RBAC) settings. Azure AI Foundry requires appropriate permissions to deploy models. Please ensure that your user account is assigned either the Owner or Contributor role for the Azure subscription.

For more details on required permissions, you can refer: Fine-tune models using serverless APIs in Azure AI Foundry.

Additionally, the error message indicates a potential private endpoint (PE) or authentication issue related to accessing storage for your fine-tuning run. Please verify that your storage is correctly linked and that the identity used for deployment has the required permissions, such as the Storage Blob Data Reader role.

You can also try using: Fine-tune models using managed compute (preview).

I hope you understand! Thank you.