Edit

Run batch predictions using Azure Machine Learning designer

  • Article

In this article, you learn how to use the designer to create a batch prediction pipeline. Batch prediction lets you continuously score large datasets on-demand using a web service that can be triggered from any HTTP library.

In this how-to, you learn to do the following tasks:

  • Create and publish a batch inference pipeline
  • Consume a pipeline endpoint
  • Manage endpoint versions

To learn how to set up batch scoring services using the SDK, see the accompanying tutorial on pipeline batch scoring.

Prerequisites

This how-to assumes you already have a training pipeline. For a guided introduction to the designer, complete part one of the designer tutorial.

Important

If you do not see graphical elements mentioned in this document, such as buttons in studio or designer, you may not have the right level of permissions to the workspace. Please contact your Azure subscription administrator to verify that you have been granted the correct level of access. For more information, see Manage users and roles.

Create a batch inference pipeline

Your training pipeline must be run at least once to be able to create an inferencing pipeline.

  1. Go to the Designer tab in your workspace.

  2. Select the training pipeline that trains the model you want to use to make prediction.

  3. Submit the pipeline.

Submit the pipeline

Screenshot showing the set up pipeline job with the experiment drop-down and submit button highlighted.

You'll see a submission list on the left of canvas. You can select the job detail link to go to the job detail page, and after the training pipeline job completes, you can create a batch inference pipeline.

Screenshot showing the submitted job list.

  1. In job detail page, above the canvas, select the dropdown Create inference pipeline. Select Batch inference pipeline.

    Note

    Currently auto-generating inference pipeline only works for training pipeline built purely by the designer built-in components.

    Screenshot of the create inference pipeline drop-down with batch inference pipeline highlighted.

    It will create a batch inference pipeline draft for you. The batch inference pipeline draft uses the trained model as MD- node and transformation as TD- node from the training pipeline job.

    You can also modify this inference pipeline draft to better handle your input data for batch inference.

    Screenshot showing a batch inference pipeline draft.

Add a pipeline parameter

To create predictions on new data, you can either manually connect a different dataset in this pipeline draft view or create a parameter for your dataset. Parameters let you change the behavior of the batch inferencing process at runtime.

In this section, you create a dataset parameter to specify a different dataset to make predictions on.

  1. Select the dataset component.

  2. A pane will appear to the right of the canvas. At the bottom of the pane, select Set as pipeline parameter.

    Enter a name for the parameter, or accept the default value.

    Screenshot of cleaned dataset tab with set as pipeline parameter checked.

  3. Submit the batch inference pipeline and go to job detail page by selecting the job link in the left pane.

Publish your batch inference pipeline

Now you're ready to deploy the inference pipeline. This will deploy the pipeline and make it available for others to use.

  1. Select the Publish button.

  2. In the dialog that appears, expand the drop-down for PipelineEndpoint, and select New PipelineEndpoint.

  3. Provide an endpoint name and optional description.

    Near the bottom of the dialog, you can see the parameter you configured with a default value of the dataset ID used during training.

  4. Select Publish.

Screenshot of set up published pipeline.

Consume an endpoint

Now, you have a published pipeline with a dataset parameter. The pipeline will use the trained model created in the training pipeline to score the dataset you provide as a parameter.

Submit a pipeline job

In this section, you'll set up a manual pipeline job and alter the pipeline parameter to score new data.

  1. After the deployment is complete, go to the Endpoints section.

  2. Select Pipeline endpoints.

  3. Select the name of the endpoint you created.

Screenshot of the pipeline endpoint tab.

  1. Select Published pipelines.

    This screen shows all published pipelines published under this endpoint.

  2. Select the pipeline you published.

    The pipeline details page shows you a detailed job history and connection string information for your pipeline.

  3. Select Submit to create a manual run of the pipeline.

    Screenshot of set up pipeline job with parameters highlighted.

  4. Change the parameter to use a different dataset.

  5. Select Submit to run the pipeline.

Use the REST endpoint

You can find information on how to consume pipeline endpoints and published pipeline in the Endpoints section.

You can find the REST endpoint of a pipeline endpoint in the job overview panel. By calling the endpoint, you're consuming its default published pipeline.

You can also consume a published pipeline in the Published pipelines page. Select a published pipeline and you can find the REST endpoint of it in the Published pipeline overview panel to the right of the graph.

To make a REST call, you'll need an OAuth 2.0 bearer-type authentication header. See the following tutorial section for more detail on setting up authentication to your workspace and making a parameterized REST call.

Versioning endpoints

The designer assigns a version to each subsequent pipeline that you publish to an endpoint. You can specify the pipeline version that you want to execute as a parameter in your REST call. If you don't specify a version number, the designer will use the default pipeline.

When you publish a pipeline, you can choose to make it the new default pipeline for that endpoint.

Screenshot of set up published pipeline with set as default pipeline for this endpoint checked.

You can also set a new default pipeline in the Published pipelines tab of your endpoint.

Screenshot of sample pipeline tab with set as default highlighted.

Update pipeline endpoint

If you make some modifications in your training pipeline, you may want to update the newly trained model to the pipeline endpoint.

  1. After your modified training pipeline completes successfully, go to the job detail page.

  2. Right click Train Model component and select Register data

    Screenshot of the train model component options with register data highlighted.

    Input name and select File type.

    Screenshot of register as data asset with new data asset selected.

  3. Find the previous batch inference pipeline draft, or you can just Clone the published pipeline into a new draft.

  4. Replace the MD- node in the inference pipeline draft with the registered data in the step above.

    Screenshot of updating the inference pipeline draft with the registered data in the step above.

  5. Updating data transformation node TD- is the same as the trained model.

  6. Then you can submit the inference pipeline with the updated model and transformation, and publish again.

Next steps