Custom Event Trigger for multiple synapse pipelines

Question

Hi everyone,

We have a process in place which sends and event to an event grid once a staging table is loaded. The payload of the event includes the table name.

The next step is to develop a trigger which executes the correspondent pipeline to move the data from staging (delta tables in ADLS) to the DWH (Synpase dedicated pool).

I can see at the moment some options:

• Create one trigger for every pipeline. The major drawback is the increasing number of triggers.
• Implement some intelligence based on the payload and start only the correspondent pipeline. I don't know if this is possible.
• Create a parent pipeline which execute all the pipelines depending on the payload of the event that activates the trigger. I don't know if this is feasible.

In your experience, what would be the best way to implement this requirement.

Best regards,

Paul

Answer 1

Hi @Paul Hernandez ,

Thank you for posting query in Microsoft Q&A Platform.

Best option is creating a parent or master pipeline and then use Execute pipeline activity inside it to run child or actual pipelines. In this way you no need to have so many triggers.

Hope this will help. Please let us know if any further queries.

------------

Please consider hitting Accept Answer button. Accepted answers helps community as well.