Azure Data Factory Pipeline stuck in queue staus

Question

I am running a pipeline to move data into an SQL database. The data flow transformation gets stuck in the queue status and does not progress. I have checked , and cannot seem to find the issue. The pipeline Run ID is 93748757-830d-4df5-9354-4f6351248220. Please assist in resolving this.

Answer 1

Hello TobyE,

Welcome to the MS Q&A platform.

We have the below Microsoft documentation to troubleshoot if the pipeline is queued or stuck for a long time.

https://learn.microsoft.com/en-us/azure/data-factory/pipeline-trigger-troubleshoot-guide#pipeline-status-is-queued-or-stuck-for-a-long-time

<Copied from the documentation page>

Here are the possible root Causes

This can happen for various reasons like hitting concurrency limits, service outages, network failures and so on.

Resolution

Concurrency Limit: If your pipeline has a concurrency policy, verify that there are no old pipeline runs in progress.

Monitoring limits: Go to the ADF authoring canvas, select your pipeline, and determine if it has a concurrency property assigned to it. If it does, go to the Monitoring view, and make sure there's nothing in the past 45 days that's in progress. If there is something in progress, you can cancel it and the new pipeline run should start.

• Transient Issues: It is possible that your run was impacted by a transient network issue, credential failures, services outages etc. If this happens, Azure Data Factory has an internal recovery process that monitors all the runs and starts them when it notices something went wrong. You can rerun pipelines and activities as described here.. You can rerun activities if you had canceled activity or had a failure as per Rerun from activity failures. This process happens every one hour, so if your run is stuck for more than an hour, create a support case.

If you are using SHIR in your activity, please change it to Azure IR and see it helps.

I hope this helps. Please let me know if you have any further questions.