Schedule two triggers for a single pipeline, one every 30 mins and the other once a day at 11 am

Question

How can I schedule a pipeline in Azure Synapse Analytics to run every 30 mins and once a day at 11 am? I've already created two triggers: a tumbling windows trigger for every 30 mins and a once-a-day trigger for 11 am. However, I need to prevent the two triggers from running simultaneously. If the 30-min trigger is running, I want the 11 am trigger to wait for it to finish before running, and vice versa. How can I handle this situation?

Thankyou!

Answer 1

Hi ,

Thanks for reaching out to Microsoft Q&A.

1: Create the Triggers

1. Tumbling Window Trigger: Create a tumbling window trigger that runs every 30 minutes. This type of trigger is designed to manage concurrency effectively.
2. Daily Trigger: Create a schedule trigger that runs once a day at 11 AM.

2.1: Configure Concurrency

To prevent the two triggers from running simultaneously, you need to set the concurrency limit for the pipeline:

• Set Max Concurrency: In the pipeline settings, set the Max Concurrency to 1. This means that if one instance of the pipeline is running, the other instance (whether triggered by the 30-minute or the daily trigger) will wait until the current instance finishes before starting.

(OR)

2.2: Add Control Flow Logic in Your Pipeline:

• Use a control table (ex: in azure SQL database) or an azure storage blob to manage the execution state of the pipeline.
• Implement locking mechanisms to prevent simultaneous runs.

Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.