Endpoints for getting metadata about published models?

Question

My question centers around working with AML models that have been published as web services, as described here:
https://learn.microsoft.com/en-us/azure/machine-learning/how-to-consume-web-service?tabs=azure-portal

Are there any endpoints or ways of obtaining more detailed information about a published model? For example, the documentation states that inputs to the model are passed in via a "data" property, and obviously, this will vary my the model:

{
"data":
[
<model-specific-data-structure>
]
}

Is there a way to programatically find out what the model expects as input?

The full 'wish-list' of metadata info we'd like is listed here:

• What models are available for serving
• What is the model prediction endpoint
• What are the required inputs and their data types
• What are the model outputs and data types

Are there any endpoints or any way at getting to this information?

Answer 1

Hello @MK RP

Thanks for reaching out to us, I will answer your question below, at the meantime, if you feel like I am not getting your point well, please point it out and correct me.

I think you are mentioning how to monitor published model and collect data, there are several choice depends on the data you want to collect:

1. Collect data from models in production - https://learn.microsoft.com/en-us/azure/machine-learning/how-to-enable-data-collection

The following data can be collected:

Model input data from web services deployed in an AKS cluster. Voice audio, images, and video are not collected.
Model predictions using production input data.

Once collection is enabled, the data you collect helps you:

Monitor data drifts on the production data you collect.
Analyze collected data using Power BI or Azure Databricks
Make better decisions about when to retrain or optimize your model.
Retrain your model with the collected data.

2 . Monitor and collect data from ML web service endpoints - https://learn.microsoft.com/en-us/azure/machine-learning/how-to-enable-app-insights

You can use Azure Application Insights to collect the following data from an endpoint:

Output data
Responses
Request rates, response times, and failure rates
Dependency rates, response times, and failure rates
Exceptions

3 . More details from Data Drift - https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-datasets?tabs=python

With Azure Machine Learning dataset monitors (preview), you can:

Analyze drift in your data to understand how it changes over time.
Monitor model data for differences between training and serving datasets. Start by collecting model data from deployed models.
Monitor new data for differences between any baseline and target dataset.
Profile features in data to track how statistical properties change over time.
Set up alerts on data drift for early warnings to potential issues.
Create a new dataset version when you determine the data has drifted too much.

Hope above information helps, please let us know if you need further helps!

Regards,
Yutong

-Please kindly accept the answer if you feel helpful, thanks a lot.