Any Limitation in tumbling window trigger

Question

Any Limitation in tumbling window trigger

Karthikeyan Shivasankaran 21

In ADF, We have first few pipelines run in sequence with dependencies and branches out at the end. Having said that, its our first experience to opt for tumbling window trigger option. is there any limitation in this option? what are the other options available for scheduling Data factory Pipelines in Azure.

Karthikeyan Shivasankaran 21 Reputation points

2021-03-19T10:57:04.233+00:00
Appreciate if I get this scenario answered,

There are multiple master pipelines should be triggered with dependencies.
The First one leading the queue is significant and has this criteria,

It has to check for a file arrival in FTP once in two weeks say every 15th and 30th of the month.

The file is not guaranteed to be exactly available on 15th/30th of every month, it could be 14 or 16th for 15th schedule and similarly for 30th.

Rest of the Trigger should follow the First event based trigger.

if any failure among triggers, either re-run or cancel the entire cycle (eg., 15th cycle)
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2021-03-30T01:11:12.693+00:00

Hi anonymous user,

Sorry for the delay in response. Since there is no guarantee of the file arrival, I would suggest you to please check on the Schedule trigger option. You can start the schedule from 14th to 17th and similar for 30th to trigger your master pipeline once a day, and in the very beginning of the flow, have a validation activity to check if the file has arrived, if the file is arrived continue executing the flow, if not just end the flow. In continuation flow have execute pipeline activities in it to trigger the dependent pipeline in order once the prior activity is completed successfully. If any activity of the pipeline fails, then the subsequent flow is discontinued and the pipeline status is marked as failed.
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2021-03-30T01:14:07.26+00:00

Continuation to above comment:

In case if you want to re-run from the failed activity, you can always go to Monitor section in ADF UI then to Pipeline runs and then click select the failed pipeline and you see an option Rerun from failed activity

Hope this info helps.

Thanks
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2021-03-30T18:58:51.313+00:00

Hi there,

Following up to see if the above information was helpful? Do let us know if you have further query.

Thanks

1 answer

Your answer

Karthikeyan Shivasankaran 21 Reputation points

2021-03-19T10:57:04.233+00:00

Appreciate if I get this scenario answered,

There are multiple master pipelines should be triggered with dependencies.
The First one leading the queue is significant and has this criteria,

It has to check for a file arrival in FTP once in two weeks say every 15th and 30th of the month.

The file is not guaranteed to be exactly available on 15th/30th of every month, it could be 14 or 16th for 15th schedule and similarly for 30th.

Rest of the Trigger should follow the First event based trigger.

if any failure among triggers, either re-run or cancel the entire cycle (eg., 15th cycle)
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2021-03-30T01:11:12.693+00:00

Hi anonymous user,

Sorry for the delay in response. Since there is no guarantee of the file arrival, I would suggest you to please check on the Schedule trigger option. You can start the schedule from 14th to 17th and similar for 30th to trigger your master pipeline once a day, and in the very beginning of the flow, have a validation activity to check if the file has arrived, if the file is arrived continue executing the flow, if not just end the flow. In continuation flow have execute pipeline activities in it to trigger the dependent pipeline in order once the prior activity is completed successfully. If any activity of the pipeline fails, then the subsequent flow is discontinued and the pipeline status is marked as failed.
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2021-03-30T01:14:07.26+00:00

Continuation to above comment:

In case if you want to re-run from the failed activity, you can always go to Monitor section in ADF UI then to Pipeline runs and then click select the failed pipeline and you see an option Rerun from failed activity

Hope this info helps.

Thanks
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2021-03-30T18:58:51.313+00:00

Hi there,

Following up to see if the above information was helpful? Do let us know if you have further query.

Thanks

Answer 1

Hi anonymous user,

Welcome to Microsoft Q&A forum and thanks for your query.

Tumbling window triggers are a type of trigger that fires at a periodic time interval from a specified start time (basically when you want to process a time based data), while retaining state. Tumbling windows are a series of fixed-sized, non-overlapping, and contiguous time intervals.

This triggers offer a suite of features for complex scenarios like :

Here are few Limitations:

A tumbling window trigger has a one-to-one relationship with a pipeline and can only reference a singular pipeline.
If one of the dependencies triggers fails, you must successfully rerun it in order for the dependent trigger to run.
A tumbling window trigger will wait on dependencies for seven days before timing out. After seven days, the trigger run will fail

ADF also has another time based trigger called Schedule trigger. To know more in detail about the differences between a Schedule trigger and a Tumbling window trigger, please refer to this doc: Trigger type comparison

Related helpful docs: Tumbling window trigger Usage scenarios and examples

Hope this info helps.

----------

Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.

KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2021-03-18T19:49:20.007+00:00

Hi anonymous user,

Just checking in to see if the above suggestion was helpful. If this answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Thanks

Share via

Any Limitation in tumbling window trigger

1 answer

Your answer