ADF pipeline scheduling: different frequency based on different week and month

Arundhati Sen 151 Reputation points
2023-06-14T13:42:17.5+00:00

I have a requirement to schedule my pipeline as below:

  1. Hourly, during the first three weeks of a quarter (first three weeks of January, April, July, October) 
  2. Daily once, on other days

Please suggest me how do I create the schedule trigger. How do I set the frequency and recurrence to achieve above?
Also, I can use multiple triggers to achieve this.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,674 questions
{count} votes

1 answer

Sort by: Most helpful
  1. QuantumCache 20,371 Reputation points Moderator
    2023-06-19T20:28:17.9566667+00:00

    Hello @Arundhati Sen, Your approach looks good, did you face any issues in the Schedule?
    As i have also verified this info on Scheduling, seems it is not available out-of-the-box as of now,

    But we can certainly use combination of Triggers schedule and then Skip the Run in the Pipeline if the current day is a weekend instead of weekday, please do let me know if the below answer helps!

    Azure Data Factory doesn't directly provide a way to exclude a specific time (like 2PM). However, you can do it indirectly by setting multiple schedules within the same trigger for different time slots (4AM to 12PM and 4PM to 10PM).

    1. You'd create a second trigger for daily execution. This can be a Schedule Trigger that recurs daily and is scheduled to run at 2PM.
    {
        "name": "DailyTrigger",
        "properties": {
            "runtimeState": "Started",
            "pipelines": [
                {
                    "pipelineReference": {
                        "referenceName": "<your-pipeline-name>",
                        "type": "PipelineReference"
                    }
                }
            ],
            "type": "ScheduleTrigger",
            "typeProperties": {
                "recurrence": {
                    "frequency": "Day",
                    "interval": 1,
                    "startTime": "2023-01-01T14:00:00Z",
                    "endTime": "2023-12-31T14:00:00Z",
                    "timeZone": "UTC"
                }
            }
        }
    }
    

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.