Azure Logic Apps
An Azure service that automates the access and use of data across clouds without writing code.
3,542 questions
How to trigger an ADF based on any data changes within a table in database
Nandan Hegde
36,146
Reputation points MVP
Volunteer Moderator
Hello All,
Is it possible to trigger an ADF pipeline after some data changes have been made in a table within a database ?
We are using MDS for master data management and want to push the data into the Azure SQL Datawarehouse the moment someone updates/inserts data into MDS.
So is there any way to auto trigger the ADF job the moment someone publishes/saves data into MDS?
Azure Logic Apps
Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
719 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,624 questions
SQL Server Migration Assistant
SQL Server Migration Assistant
A Microsoft tool designed to automate database migration to SQL Server from Access, DB2, MySQL, Oracle, and SAP ASE.
568 questions
{count} vote
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 26%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HimanshuSinha-msft 19,486 Reputation points Microsoft Employee Moderator2020-12-15T01:35:02.773+00:00 Hello @Nandan Hegde
Thanks for the ask and using the forum .I think we can always have a trigger run scheduled which can check all the tables in MDS and then update the the sink tables .
When there is no update or insert the trigger runs and does nothing . We can also implement this using change tracking ( I am assuming that you have SQL based MDS ) . I
Lets say that you have the trigger scheduled every 15 mins and it runs at 9:00 AM , 9:15 AM ... .-----------
Comment 1/2 ------------------------ -
HimanshuSinha-msft 19,486 Reputation points Microsoft Employee Moderator
2020-12-15T01:35:50.973+00:00 -----------
Comment 2/2 ------------------------Best case scenario : We update the data at 9:14 AM and the very next minute ADF triggers the pipeline and start the sync .
Worst case scenario : We update the data at 9:01 AM and now we need to wait for 14 mins before the trigger to fire and updates to complete .
The ask is can you guys wait for 15 mins ? If yes then all ADF / Logic apps / and ASA can solve the problem ,
if not then we can think to call the ADF REST API for every update/ Insert . I dont know if any service does this unless we write something oif our known .https://learn.microsoft.com/en-us/rest/api/datafactory/pipelines/createrun
Thanks
Himanshu -
HimanshuSinha-msft 19,486 Reputation points Microsoft Employee Moderator
2020-12-17T22:57:34.69+00:00 Hello @Nandan Hegde ,
We have not heard back from you on this and was just following up .
Incase if you have resolution , request you to share the same here , so that other community members can benefit from that .
Thanks
Himanshu -
HimanshuSinha-msft 19,486 Reputation points Microsoft Employee Moderator
2020-12-23T20:15:52.047+00:00 Hello @Nandan Hegde ,
We have not heard back from you on this yet and was just following up .
Incase if you have resolution , request you to share the same here , so that other community members can benefit from that .
Thanks
Himanshu -
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 57.00000000000001%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESJ%3C/text%3E%3C/svg%3E)
Sonam Jain 1 Reputation point Microsoft Employee2021-03-17T05:29:47.097+00:00 I have a similar requirement as follows- I have enabled Change Data Capture on Azure SQL tables and want to trigger ADF pipeline as soon as there is an entry into CDC table. Can you please help how this can be achieved in Data Factory.
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 13%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDM%3C/text%3E%3C/svg%3E)
DJ Mata 1 Reputation point2021-04-18T04:41:51.96+00:00 Hi Nandan ,
Wanted to check if you were able to achieve this .
I have exact same situation and I am keen to know how you solved this .
Thanks in advance. -
DJ Mata 1 Reputation point
2021-04-18T04:42:49.463+00:00 Hi @HimanshuSinha-msft
Your approach results in lot of compute power wasted .
Is there a better way . -
DJ Mata 1 Reputation point
2021-04-18T05:22:35.13+00:00 Hi @Nandan Hegde
Wanted to check if you were able to achieve this .
I have exact same situation and I am keen to know how you solved this .
Thanks in advance.
Sign in to comment
Sign in to answer