Sending Pipeline Failure Alerts via Logic App in ADF

Question

How can a pipeline failure alert be sent via a Logic App when an activity fails in Azure Data Factory? I'm trying to extract the error message to send it via email using the following pipeline expressions:

1.

@concat('Target: ', activity('raw_data_quality_pipeline')?.error?.target, ', 
Error Message: ', activity('raw_data_quality_pipeline').error.message
)

@concat('Target: ', activity('raw_data_quality_pipeline')?.error?.target, ', 
Error Message: ',replace(replace(activity('raw_data_quality_pipeline')?.error?.message, '"', ''), '\n', ' ')
)

Both approaches result in the following error messages:

-

{ "errorCode": "BadRequest", "message": "Operation on target email_notifications failed: {\"error\":{\"code\":\"InvalidRequestContent\",\"message\":\"The request content is not valid and could not be deserialized: 'After parsing a value an unexpected character was encountered: S. Path 'Message', line 28, position 75.'.\"}}", "failureType": "UserError", "target": "notification_pipeline_raw_data_quality_alerts", "details": "" }

Operation on target email_notifications failed: {"error":{"code":"InvalidRequestContent","message":"The request content is not valid and could not be deserialized: 'Bad JSON escape sequence: \\\n. Path 'Message', line 40, position 45.'."}}

What steps can be taken to resolve this issue?

Answer 1

Hi Aditya Singh,

Thanks for reaching out to Microsoft Q&A.

The errors you're encountering stem from special characters in the error message that are causing JSON parsing issues when the message is sent to the Logic App. Specifically, characters like newline (\n), carriage return (\r), backslash (\), and double quotes (") can break the JSON structure if not properly escaped or encoded.

To resolve this issue, you can use the encodeUriComponent() function in ADF to encode the error message. This function replaces special characters with their URI-encoded equivalents, ensuring that the JSON payload remains valid when sent to the Logic App.

You can try modifying your expression similar to below:

@concat(
    'Target: ',
    activity('raw_data_quality_pipeline')?.error?.target,
    ', Error Message: ',
    encodeUriComponent(activity('raw_data_quality_pipeline')?.error?.message)
)

Steps to try:

1. Update the ADF Pipeline Expression: Replace your existing expression with the one provided above. The encodeUriComponent() function will encode special characters in the error message.
2. Modify the Logic App to Decode the Error Message: In your Logic App, use the decodeUriComponent() function to decode the error message back to its original form before including it in the email notification. Here's how you can do it:
   • When you receive the error message in the Logic App (probably as part of the HTTP request body), apply the decodeUriComponent() function to it. For ex: if you're using the value in an email body, you can set the email content to:

@{decodeUriComponent(triggerBody()?['ErrorMessage'])}

Replace 'ErrorMessage' with the actual key you're using in the HTTP request body.

Explanation:

• Why encodeUriComponent() Helps: This function ensures that all special characters are percent encoded, which makes the error message safe to include in JSON payloads without causing parsing errors. Handling in Logic App: By decoding the error message in the Logic App, you retrieve the original error message content, which can then be included in the email body.

If you prefer not to encode and decode the error message, you can remove problematic characters using the replace() function multiple times. However, this approach is not reliable because you might miss some special characters. The encoding method is more robust and recommended & this method may still result in errors if there are other special characters in the error message.

Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.

Answer 2

Hello Aditya Singh,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you are sending Pipeline Failure Alerts via Logic App in ADF.

Alternative, to the above solution by @Vinodh247 to avoids the need for encoding and decoding error messages. You can handle this error using "On Failure" or "On Completion" conditions in Azure Data Factory.

Then, convert the error message to a string and use a simple @string() function to avoid the need for escaping characters. This ensures that the error message remains readable and avoids JSON parsing issues.

 @concat(
       'Target: ',
       activity('raw_data_quality_pipeline')?.error?.target,
       ', Error Message: ',
       string(activity('raw_data_quality_pipeline')?.error?.message)
   )

https://learn.microsoft.com/en-us/azure/data-factory/control-flow-web-activity

Next is to send the custom error message to a Logic App using a Web Activity in ADF. The Web Activity can trigger the Logic App directly, passing a well-formatted payload, which you can then use for email notifications or other actions. For example, payload sent to the Logic App:

   {
       "Target": "@{activity('raw_data_quality_pipeline')?.error?.target}",
       "ErrorMessage": "@{string(activity('raw_data_quality_pipeline')?.error?.message)}"
   }

https://learn.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions

The step 4 is to clean up if you still encounter special character issues (such as line breaks or double quotes), you can use the replace() function in ADF to clean up the message. For example:

   @replace(replace(string(activity('raw_data_quality_pipeline')?.error?.message), '"', ''), '\n', ' ')

Lastly, in the Logic App, use the data passed from the ADF Web Activity to send a well-formatted email notification. You don’t need to decode anything, as the error message will already be cleaned and formatted from ADF.

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.