Call Synapse pipeline with a notebook activity
APPLIES TO: Azure Data Factory Azure Synapse Analytics
Tip
Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!
In this tutorial, you create an end-to-end pipeline that contains the Web, Until, and Fail activities in Azure Data Factory.
Web calls a Synapse pipeline with a notebook activity.
Until gets Synapse pipeline status until completion (status output as Succeeded, Failed, or canceled).
Fail fails activity and customizes both error message and error code.
Prerequisites
- You have a Synapse pipeline with a notebook activity.
- Use role-based access control(RBAC) to grant permissions for Data Factory to call Synapse workspace endpoint, details refer to Managed Identity.
- Grant permissions for Data Factory to execute/run Synapse pipeline, details refer to Synapse RBAC roles.
How to use this template
Go to the Call Synapse pipeline with a notebook activity template.
Select Use this template. You'll see a pipeline created.
Pipeline introduction and configuration
Review the configurations of your pipeline and make any necessary changes.
Pipeline parameters. Change settings if necessary.
SynapseEndpoint: The workspace development endpoint, for example https://
myworkspace
.dev.azuresynapse.net.PLName: Synapse pipeline name in Synapse workspace to execute.
FailedOrCancelledErrorCode: Customize error code if Synapse pipeline fails.
A Web activity to call Synapse pipeline.
URL:
@Concat(pipeline().parameters.SynapseEndpoint,'/pipelines/',pipeline().parameters.PLName,'/createRun?api-version=2020-12-01')
. Refer to Create pipeline run via Synapse REST API.Authentication: Managed Identity is used for authentication. Refer to Prerequisites to grant necessary permissions.
Resource :
https://dev.azuresynapse.net/
.
Do Web activity to get Synapse pipeline status, Until web activity returns status of Succeeded, Failed, or Canceled.
Web URL:
@concat(pipeline().parameters.SynapseEndpoint,'/pipelineruns/',activity('Call Synapse Pipeline with Notebook').output.runId,'?api-version=2020-12-01')
. Refer to Synapse Get Pipeline Run REST API.Until expression:
@or(or(equals('Succeeded',activity('Get run status').output.status),equals('Failed',activity('Get run status').output.status)),equals('Cancelled',activity('Get run status').output.status))
.
An If Condition activity evaluates pipeline run status.
- Expression":
@equals('Failed',activity('Get run status').output.status)
- Expression":
A Fail activity to fail the pipeline.
Fail message:
@activity('Get run status').output.message
Error code:
@pipeline().parameters.FailedOrCancelledErrorCode