Azure OpenAI model monitoring does not provide evaluation metrics - Job Status "Queued"

ferrand 55 Reputation points
2024-03-06T22:56:24.43+00:00

I am using Azure Machine Learning Studio as it provides more possibilities in terms of customization of the monitoring. Model is in production but so far I am sending questions only in the test tab of the endpoint in Azure Studio. I have sent 50 questions in this test tab as suggested by Microsoft.

I followed step by step this official Microsoft guide: https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/how-to-monitor-generative-ai-applications?view=azureml-api-2

I also watched this official tutorial: https://www.youtube.com/watch?v=hVsjcsoGAww

However, after deploying my flow to an endpoint and creating the monitoring I only get this:
Screenshot 2024-03-06 at 23.44.29

It gets stuck here:User's image

The model data collector with inputs and outputs seems to work. Sampled files are being collected.

If I click on the jobs related to the output I see that the Status is on "Queued" with the message "Information: This job is waiting for available quota in scheduling queue."
Screenshot 2024-03-06 at 23.49.34

How can I proceed? Many thanks in advance for your kind support.

Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
3,336 questions
Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
4,085 questions
Azure AI services
Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
3,604 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.