is there any in built Alert functionality for adls containers in case of pipelines having json files / folders are not updated

Question

Hi,

We have created few Pipelines in Azure Datalake Storage to fetch data from Plant level softwares to the Azure datalake container storage account.
data is pushed from source in json file folders at every 02 mins.
in case of any issue from Source, if the file folders stops updating, we do not get any alert / mail.
hence we are monitoring the pipelines manually.

is there any in built functionality in Azure cloud, so that i can automate things to get alerts in case json file folders are not updated.

refer below screen shot for reference:

Regards
Satish

Answer 1

Hello @Satish Kumar ,
Thanks for the question and using MS Q&A platform.
As we understand the ask here is how to setup an alert when the latest file does not arrive in the storage container , please do let us know if its not accurate.
You can use Logic apps and also Azure data factory( ADF ) to implement the alert . I can walk you through the ADF imapmentation .
You can use the Getmadata activity to read the files metadata and create the logic . if the logic does not comply , you can fail the pipeline . You will have create a alert of failures and you should recieve an email .
Please do read more on this here : https://learn.microsoft.com/en-us/azure/data-factory/control-flow-get-metadata-activity
https://learn.microsoft.com/en-us/azure/data-factory/monitor-metrics-alerts

Please do let me if you have any queries.
Thanks
Himanshu

---

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators