How to set 'DependsOn' setting dynamically in copy activity in Data Factory

Question

I am trying to set 'DependsOn' property in copy activity using parameters. Is there any way to do it ?

Answer 1

Not sure if it is possible to make dependsOn dynamic but see if you can use IF condition activity and then use Copy activity in it based on condition.

---

If the response helped, do "Accept Answer" and upvote it - Vaibhav

Answer 2

Adding IF activity defeats the purpose of limiting total number of activities at 40. have a master ETL pipeline that loads more than 80 tables currently (will increase in near future to be around 160). Currently I have bundled all the activities into stages based on their dependencies and used the bundled pipelines in the master to limit them below 40 activities. However, if I could set 'DependsOn' dynamically, then I can use 'Foreach' to load each table (activity) easily with significantly less number of activity in a single master pipeline. Hope I am explain my issue here. Please let me know if there is a solution for it.

@MalleswaraReddy : I have the pipelines created, just that I want to be able to use Foreach to load all the tables with their dependencies.

Thanks.

Answer 3

I do not fully understand you situation, but it sounds like you need more options for controlling flow and logic. I have many tricks that may or may not be helpful. I will share one below.
If I had a diagram of the logic you want to implement, I might be able to suggest a more manageable setup.

I use the below pattern when I I want to branch my control flow, without using an If Activity.

I intentionally cause the set variable activity to fail using an expression like @if(equals(pipeline().parameters.DoSucces,true),true,'IMakeError')
Then I use the dependency conditions to branch my flow. The highlighted 'skipped' dependency is important. Without the highlighted skipped dependency, the pipeline will report failure status.