Share via

Azure Data Factory pipelines stuck on first activity for 7+ hours

Marcus Ghebrehiwet 0 Reputation points
2024-04-30T10:04:21.0433333+00:00

Hi, I just woke up to monitor all the runs, and suddenly saw that all of my runs were stuck on the first activity, or not even starting the first activity some for more than 7+ hours. This happened over-night as yesterday everything was fine and I didn't change nothing that could relate to such a problem, I never faced such a problem before, I checked and there are no running processes from 30-45 days ago or such.

Screenshot_95

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,684 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. PRADEEPCHEEKATLA-MSFT 78,986 Reputation points Microsoft Employee
    2024-04-30T13:38:00.18+00:00

    @Marcus Ghebrehiwet - Thanks for the question and using MS Q&A platform.

    I'm sorry to hear that you're experiencing issues with your Azure Data Factory pipelines. Based on the information you provided, it seems like there could be a few different reasons why your pipelines are stuck.

    • One possible cause could be hitting concurrency limits. If your pipeline has a concurrency policy, verify that there are no old pipeline runs in progress. Another possibility is that there may be monitoring limits in place. You can go to the ADF authoring canvas, select your pipeline, and determine if it has a concurrency property assigned to it. If it does, go to the Monitoring view, and make sure there's nothing in the past 45 days that's in progress. If there is something in progress, you can cancel it and the new pipeline run should start.
    • It is possible that your run was impacted by a transient network issue, credential failures, services outages, etc. If this happens, Azure Data Factory has an internal recovery process that monitors all the runs and starts them when it notices something went wrong. You can rerun pipelines and activities as described here: Rerun pipelines and activities. This process happens every one hour, so if your run is stuck for more than an hour, create a support case.

    For more details, refer to Azure Data Factory: Pipeline status is queued or stuck for a long time.

    If none of these solutions work, please provide more information about your pipeline and the activities involved and ADF region so that we can investigate further.

    Hope this helps. Do let us know if you any further queries.