Send an email with an Azure Data Factory or Azure Synapse pipeline

APPLIES TO: Azure Data Factory Azure Synapse Analytics

It's often necessary to send notifications during or after execution of a pipeline. Notification provides proactive alerting and reduces the need for reactive monitoring to discover issues.  This article shows how to configure email notifications from an Azure Data Factory or Azure Synapse pipeline. 

Prerequisites

  • Azure subscription. If you don't have an Azure subscription, create a free account before you begin.
  • Logic App. To trigger sending an email from the pipeline, you use Logic Apps to define the workflow. For details on creating a Logic App workflow, see How to create a logic app.

Create the email workflow in your Logic App

Create a Logic App workflow named SendEmailFromPipeline. Define the workflow trigger as When an HTTP request is received, and add an action of Office 365 Outlook – Send an email (V2).

Shows the Logic App workflow designer with a Send Email (V2); action from an HTTP request trigger.

For the HTTP request trigger, provide this JSON for the Request Body JSON Schema:

{
    "properties": {
        "dataFactoryName": {
            "type": "string"
        },
        "message": {
            "type": "string"
        },
        "pipelineName": {
            "type": "string"
        },
        "receiver": {
            "type": "string"
        }
    },
    "type": "object"
}

The HTTP Request in the Logic App Designer should look like this:

Shows the Logic App workflow designer for the HTTP Request trigger with the Request Body JSON Schema field populated.

For the Send Email (V2) action, customize how you wish to format the email, using the properties from the request Body JSON schema:

Shows the Logic App workflow designer for the Send Email (V2) action.

Save the workflow. Browse to the Overview page for the workflow. Make a note of the Workflow URL for your new workflow then, highlighted in the image below:

Shows the Logic App workflow Overview tab with the Workflow URL highlighted.

Note

To find the Workflow URL you must browse to the workflow itself, not just the logic app that contains it. From the Workflows page of your logic app instance, choose the workflow and then navigate to its Overview page.

Create a pipeline to trigger your Logic App email workflow

Once you create the Logic App workflow to send email, you can trigger it from a pipeline using a Web activity.

  1. Create a new pipeline and find the Web activity under the General category, to drag it onto the editing canvas.

  2. Select the new Web1 activity, and then select the Settings tab.

    Provide the URL from the Logic App workflow you created previously in the URL field.

    Provide the following JSON for the Body:

       {
        "message" : "This is a custom dynamic message from your pipeline with run ID @{pipeline().RunId}.",
        "dataFactoryName" : "@{pipeline().DataFactory}", 
        "pipelineName" : "@{pipeline().Pipeline}", 
        "receiver" : "@{pipeline().parameters.receiver}"
       }
    

    Use dynamic expressions to generate useful messages for events in your pipelines. Notice that the JSON format here matches the JSON format you defined in the Logic App, and you can also customize these as required.

    Shows a pipeline with a Web activity configured with the Logic App workflow URL and JSON message body.

  3. Select the background area of the pipeline designer to select the pipeline properties page and add a new parameter called receiver, providing an email address as its Default value.

    In this example, we provide the receiver email from a pipeline parameter we define arbitrarily. The receiver value could be taken from any expression, or even linked data sources.

    Shows the configuration of the receiver parameter in the pipeline designer.

  4. Publish your pipeline, and then trigger it manually to confirm the email is sent as expected.

    Shows how to manually trigger the pipeline.

Add dynamic messages with system variables and expressions

You can use system variables and expressions to make your messages dynamic. For example:  

  • @activity("CopyData").output.errors[0].Message

  • @activity("DataFlow").error.Message

The above expressions will return the relevant error messages from a Copy activity failure, which can be redirected then to your Web activity that sends the email. Refer to the Copy activity output properties article for more details.

Next steps

How to send Teams notifications from a pipeline