Azure Data Factory pipeline stucks with In Progress status

Question

Hi!
I have 1 master pipeline which calls in parallel others. Alter 1 more child pipeline adding, the execution of almost all pipelines (old and new one) stuck.
Activities which stuck are mainly executes some procedures in db. These procedures do not use the same tables.
How could I identify why it stuck and how to solve it?

Thanks in advance

Answer 1

Hi @Valiantsina Kavaleuskaya ,

Thank you for posting query in Microsoft Q&A Platform.

I suspect database resource is not responding, and the activities are waiting for answer. They will wait until the Timeout property is reached.

Next step is finding root cause, look at whether the database is actually executing the requests. You are using stored procs.
Check with your database person and check the logs in the database as to what happened to the request database side. Did the request execute? Did the request get stuck? Or did the request never make it into the database?

For immediate relief to stop pipelines long running, you can set the Timeout property of the Stored Proc activities to an hour or some value which you are comfortable. The default value is 7 days, meaning that the activity waits 7 days before giving up and failing.

Hope this helps. Please let us know if any further queries.

-------------------------------

Please consider accepting answer if this helps. Accepted answer helps community as well.

Answer 2

Thanks for the suggestions!
The thing which helped me is pipeline re-creation. I think some pipeline metadata was broken since it was created as a copy of some other one.

Anyway thank you very much!