Azure ML Workspace Labelling Project Validation

Question

Azure ML Workspace Labelling Project Validation

SJ Yap 11

HI, I am using Azure ML workspace Data Labelling project. I would like to know how can I trigger the Validation and Inference steps? I follow the instruction to manually labelled images, the project managed to trigger the Training steps, but there is no sign of Validation steps. I am not sure how to execute it or what is the condition to make the Validation step start. I also need to know how to trigger Inference steps as the documents I can get is limited. Any input is highly appreciated.

Thank you.

https://learn.microsoft.com/en-us/azure/machine-learning/how-to-create-labeling-projects

1 answer

Your answer

Answer 1

romungi-MSFT 49,096 Microsoft Employee Moderator

@SJ Yap The labeling projects allows user to label data with instructions and then export the same as CSV, Coco file or Azure ML dataset for use. If you are planning to consume this project with Azure ML then you could export the same as Azure ML dataset.

After exporting the project this will be available as a dataset under the dataset tab, This can be consumed from the SDK by using it as any other data.

.

If you would like to consume it from the Azure machine learning designer studio you can drag and drop the dataset on studio canvas and connect the required modules to create and run an experiment. When the experiment is run successfully you can convert it to a real time inference pipeline and then deploy it as a endpoint for inference.

Basically, the labeled data could be used as input dataset in your Azure ML experiments either through the SDK or the designer studio. I hope this helps!!

SJ Yap 11 Reputation points

2021-07-19T08:32:56.397+00:00

Hi Romungi,

Thank you so much for your detail instructions on how to consume the labelled datasets using other methods.

Just wanted to understand if by consume the other method will trigger the validation state in data label project?

Same for the Inference stage?
romungi-MSFT 49,096 Reputation points Microsoft Employee Moderator

2021-07-20T09:10:44.1+00:00
@SJ Yap I have checked internally with our team on this and based on the input these runs are triggered automatically based on following criteria.

Training - According to an internal algorithm, this run will be triggered when MLAssist believes we have enough labels to train a model or refresh current model. The required number of labels depends on number of label classes and few other factors. The estimated number of labels required to trigger the next training run are also provided.

Validation - This stage will merge into training soon.

Inference - When we have less than 300 tasks in the task queue, inference run will be triggered to generate pre-labeled tasks.

Featurization - Only happens with image multi-class or multi-label project, we use this run to generate clustered task for those images that have relatively low confidence predictions.

Apart from above to consume the labeled data you could follow the steps mentioned above with SDK and designer. These tasks are only for MLAssisted labeling and do not provide a prediction endpoint.
SJ Yap 11 Reputation points

2021-07-21T04:00:33.853+00:00

Thanks so much Romungi for the detail information regarding the ML labelling project. Just one more question, for "Validation" run, at the moment it will be auto trigger as well? In future, we should have some indication when it will be triggered as in Training stage? Your input is much appreciated.

Thank you!
romungi-MSFT 49,096 Reputation points Microsoft Employee Moderator

2021-07-22T05:20:58.883+00:00

@SJ Yap Yes, you should see more details about the stages and how they are being triggered in the documentation soon.

Share via

Azure ML Workspace Labelling Project Validation

1 answer

Your answer