Get log analytics data in a seperate table and as an alert

Question

I want to be able to get alerts about failed pipeline runs as an alert and in a separate table outside. What I currently get with azure alert is the name of my data factory and a link to the pipeline but I would like to know the name and error message of the failed pipeline, I currently can view this information in log analytics after running my query but is there a way to get the same information as an email alert and be logged to a separate database.

Answer 1

Thank you very much @ShaikMaheer-MSFT , this was very helpful. I have been able to solve the issue with logic app since I posted this question. I realized that using this method isn't very efficient because I either have to do a stored procedure activity for every step in the pipeline or do an execute pipeline activity and insert my whole pipeline process in it.
What I did with the logic app was to read the result of my log analytics query for every pipeline error and send the result as an email. I hope this is a better reply for anyone having this same issue.
I'm currently trying to do a parallel action that can also log the data to a table as well. Any help with that?

https://learn.microsoft.com/en-us/azure/azure-monitor/logs/logicapp-flow-connector

Answer 2

Hi @Adeleke Abimbola ,

Welcome to Microsoft Q&A Platform. Thank you for posting your query here.

You can create alerts on top of your query on log analytics. Click here to know more about it. To log your pipeline failures to separate database table you can have a stored procedure created in dB and call that stored procedure on pipeline activity failures to log information.

Below link shows logging pipeline executions to database table using stored procedure activity. Sample example you tweak to log only failures.
https://www.youtube.com/watch?v=yoX0wTZzIcg

To access any activity failure error message use below expression,
@activity('<activity name>').Error.message

Below few links which explains logging pipeline failures to database table.

• Azure Data Factory Pipeline Logging Error Details
• How to capture exception logs when Azure Data Factory Pipeline fail

Hope this will help. Please let us know if any further queries.

---

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators