Execute Azure Machine Learning pipelines in Azure Data Factory and Synapse Analytics
APPLIES TO:
Azure Data Factory
Azure Synapse Analytics
Tip
Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!
Run your Azure Machine Learning pipelines as a step in your Azure Data Factory and Synapse Analytics pipelines. The Machine Learning Execute Pipeline activity enables batch prediction scenarios such as identifying possible loan defaults, determining sentiment, and analyzing customer behavior patterns.
The below video features a six-minute introduction and demonstration of this feature.
Create a Machine Learning Execute Pipeline activity with UI
To use a Machine Learning Execute Pipeline activity in a pipeline, complete the following steps:
Search for Machine Learning in the pipeline Activities pane, and drag a Machine Learning Execute Pipeline activity to the pipeline canvas.
Select the new Machine Learning Execute Pipeline activity on the canvas if it is not already selected, and its Settings tab, to edit its details.
Select an existing or create a new Azure Machine Learning linked service, and provide details of the pipeline and experiment, and any pipeline parameters or data path assignments required for the pipeline.
Syntax
{
"name": "Machine Learning Execute Pipeline",
"type": "AzureMLExecutePipeline",
"linkedServiceName": {
"referenceName": "AzureMLService",
"type": "LinkedServiceReference"
},
"typeProperties": {
"mlPipelineId": "machine learning pipeline ID",
"experimentName": "experimentName",
"mlPipelineParameters": {
"mlParameterName": "mlParameterValue"
}
}
}
Type properties
Property | Description | Allowed values | Required |
---|---|---|---|
name | Name of the activity in the pipeline | String | Yes |
type | Type of activity is 'AzureMLExecutePipeline' | String | Yes |
linkedServiceName | Linked Service to Azure Machine Learning | Linked service reference | Yes |
mlPipelineId | ID of the published Azure Machine Learning pipeline | String (or expression with resultType of string) | Yes |
experimentName | Run history experiment name of the Machine Learning pipeline run | String (or expression with resultType of string) | No |
mlPipelineParameters | Key, Value pairs to be passed to the published Azure Machine Learning pipeline endpoint. Keys must match the names of pipeline parameters defined in the published Machine Learning pipeline | Object with key value pairs (or Expression with resultType object) | No |
mlParentRunId | The parent Azure Machine Learning pipeline run ID | String (or expression with resultType of string) | No |
dataPathAssignments | Dictionary used for changing datapaths in Azure Machine learning. Enables the switching of datapaths | Object with key value pairs | No |
continueOnStepFailure | Whether to continue execution of other steps in the Machine Learning pipeline run if a step fails | boolean | No |
Note
To populate the dropdown items in Machine Learning pipeline name and ID, the user needs to have permission to list ML pipelines. The UI calls AzureMLService APIs directly using the logged in user's credentials.
Related content
See the following articles that explain how to transform data in other ways: