Hello, I have been experiencing an issue, where the data factory copy activities are stuck in 'queued' status. I have followed the troubleshoot documentation but did not find anything. These are the things that I have checked: Are there old activities that are still running? --> No Try rerun failed pipeline --> Still goes to queued (for more than 1 hour) Is there an issue with the IR? --> On health check everything is green Did 'Diagnose and solve problem' find something for the queued activity? --> No We are using an Azure Integrated runtime with the following parameters: In addition, this issue showed up during a pipeline run with multiple copy activities. (It ran normally until 6:42 am and suddenly from 6:44 am it started to go to queued status). Are there any other things that I could check?

@Yoon, Sojin , all Hello all. Sorry for your experience with this issue. From my analysis I found the following: There was a service disruption on March 6. It has since been fixed. This affected Azure Data Factory, Azure Synapse Analytics, and Key Vault among other services in West Europe region . In Azure Data Factory, Copy Data may be stuck in queue or timeout and enabling interactive authoring for Azure Integration Runtime in Managed Virtual networks could fail. You can find more details in Azure Status History Tracking ID: R_36-P80 If you find the issue persists, please let us know. I hope this helps. Thank you for your patience.

Azure Data Factory copy activity stuck on queued status

Yoon, Sojin 65

Hello,

I have been experiencing an issue, where the data factory copy activities are stuck in 'queued' status.

I have followed the troubleshoot documentation but did not find anything.

These are the things that I have checked:

Are there old activities that are still running? --> No
Try rerun failed pipeline --> Still goes to queued (for more than 1 hour)
Is there an issue with the IR? --> On health check everything is green
Did 'Diagnose and solve problem' find something for the queued activity? --> No

We are using an Azure Integrated runtime with the following parameters:

User's image

In addition, this issue showed up during a pipeline run with multiple copy activities. (It ran normally until 6:42 am and suddenly from 6:44 am it started to go to queued status).

Are there any other things that I could check?

Gabriel Levaillant 35 Reputation points

2023-03-06T10:19:29.09+00:00

I have the exact same issue and I am also using the AutoResolve Integration Runtime, I did the same steps as you.

If you find any solution, I am very interested. Otherwise, it looks like this : https://social.msdn.microsoft.com/Forums/en-US/db83d241-a66b-40fc-82e0-eee13162befa/copy-activities-on-azure-ir-has-long-queue-wait-times?forum=AzureDataFactory

A service outage.

Same timeline also, everything works fine until 6:44 am UTC+1
Yoon, Sojin 65 Reputation points

2023-03-06T10:27:53.0433333+00:00

One interesting thing to note would be that I just triggered a copy activity that uses an SHIR and that activity wasn't stuck on queued (the activity still failed in the end, since it could not access the Key Vault for some reason). Maybe it really is an service outage...?
Filip Bouckaert 16 Reputation points

2023-03-06T10:39:08.9066667+00:00

We have similar issue. several copy task activities stuck as queued for hours now.
they all run on separate ADF environments.
Helle Normann 0 Reputation points

2023-03-06T10:48:44.44+00:00

We have the same problem, starting at the same time. We are running Synapse Analytics Serverless.
Matthias Delafontaine 11 Reputation points

2023-03-06T10:51:23.5633333+00:00

Same problem here. All copy data task stuck in queued status, running on the autoresolve integration runtime...
Asen Nikolov 0 Reputation points

2023-03-06T11:04:17.5266667+00:00

Same problem here. All copy data activities are stuck.
Mark Dommisse 5 Reputation points

2023-03-06T11:13:37.1633333+00:00

Same problem with Synapse Pipelines.
Filip Bouckaert 16 Reputation points

2023-03-06T11:32:20.8+00:00

apparently it is resolved (for us at least)
our processes continue.
Anonymous

2023-03-06T11:34:14.0033333+00:00

Same Problem here please fix and communicate asap.
Arthur van den Berg 0 Reputation points

2023-03-06T12:04:29.28+00:00

Workaround for me is a create a Self Hosted Integration Runtime > Azure > Select other region, eg. France Central to run. After creation adjust Linked Service to use this new created Integration Runtime.
Be aware of extra costs!
Filip Bouckaert 16 Reputation points

2023-03-06T12:21:13.5533333+00:00

after working for half an hour, they got stuck againg in queued status
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
PRADEEPCHEEKATLA-MSFT 76,921 Reputation points Microsoft Employee

2023-03-10T09:08:25.4466667+00:00

@Yoon, Sojin Glad to know that the below response was helpful. Thank you for marking this answer as accepted. This will help others find useful answers faster.

Please feel free to take a survey on the relevant answer. You can help us improve by leaving feedback verbatim.

Have a good day!

Regards,
PRADEEPCHEEKATLA-MSFT
Joost Romijn 0 Reputation points

2023-03-22T10:27:36.13+00:00

I experienced the same issue on AutoResolve Integration Runtime from 17 March until 21 March in West Europe region. In one data factory, the execution time went from +- 1 hour to 6 hours! I can't find anything about another service disruption, but it sure looks like the same issue. Any comments from Microsoft?

Edit: I just saw Jeremy Watts's reaction to the answer and the link to his thread. Looks like it was another issue that was solved yesterday.

Accepted answer

MartinJaffer-MSFT 26,021 Reputation points

2023-03-07T20:25:10.69+00:00

@Yoon, Sojin , all

Hello all. Sorry for your experience with this issue. From my analysis I found the following:

There was a service disruption on March 6. It has since been fixed. This affected Azure Data Factory, Azure Synapse Analytics, and Key Vault among other services in West Europe region. In Azure Data Factory, Copy Data may be stuck in queue or timeout and enabling interactive authoring for Azure Integration Runtime in Managed Virtual networks could fail.

You can find more details in Azure Status History Tracking ID: R_36-P80

If you find the issue persists, please let us know. I hope this helps.

Thank you for your patience.
Please sign in to rate this answer.

1 person found this answer helpful.
Petter Moe 5 Reputation points

2023-03-09T11:06:33.9266667+00:00

I'm still having the exact same problem since yesterday. Right now production data pipelines are not running as expected and causing a lot of problems for me. Region is West Europe and it seems to be only for copy data activities.

Bo Vande Sompele 0 Reputation points

2023-03-20T07:30:35.9466667+00:00

I experienced this issue as well, and indeed it was resolved. But now since 17th of March I have the same issue again: copy activity within for each, copying files from sharepoint to Blob, some files get queued for over 4 minutes and then take 5 seconds to process. The total run of the for each normally takes 50 minutes, now over 5 hours due to the queuing.

Jeremy Watts (Co-op Technology) 5 Reputation points

2023-03-21T13:02:32.8333333+00:00

Hi,

The issue appears to have started to occur again since the 16th. Please see this thread started today for more detail including my comment with examples https://learn.microsoft.com/en-us/answers/questions/1191708/data-factory-copy-data-activity-with-a-very-long-q

Alex Cruickshank 0 Reputation points

2023-06-16T13:54:23.12+00:00

Looks like this may be happening again, any copy data activities on the Autoresolveintegration runtime seem to be stuck at Queued.

Service Status shows No issues
Sign in to comment

Azure Data Factory copy activity stuck on queued status

0 additional answers