ADF Pipeline stuck in queued status

Pritpal 6

I have an azure pipeline that reads the data from the Onprem SQL server and writes onto the blob storage on Azure. We are using Integration Runtime created using our Azure virtual machine.

When I run the pipeline, it gets stuck in Queued status. Can you please let me know -

How can I access logs which will provide more details on why the process is stuck in Queue status
1. How can I resolve the issue?

KranthiPakala-MSFT 46,512 Reputation points Microsoft Employee

2020-09-30T18:59:27.97+00:00
Hi @Pritpal ,

Welcome to Microsoft Q&A forum and sorry for your experience with this issue.

As per my understanding you are using a copy activity to copy data form On prem SQL to Azure Blob Storage. Please correct me if I misunderstand your requirement.

a) If it is a Copy activity, please check out this documentation to Troubleshoot copy activity on Azure IR or SHIR
b) In case if this issue is related to source or SHIR - please check out this troubleshooting guide: Troubleshoot self-hosted integration runtime . You can also check SHIR logs as
shown below:

Continued in below comment.. (due to 1000 character limit on comments)
KranthiPakala-MSFT 46,512 Reputation points Microsoft Employee

2020-09-30T19:03:51.173+00:00
Continuation to above comment..

To troubleshoot further, could you please share below details, so that we can work with internal team to check internal logs.
a) Queued Pipeline runID & activity runID ?
b) Data Factory region
c) What is the integration runtime used for both Source and sink connection? If any SHIR is used, please confirm what version is being used? If it is AzureIR - please confirm if it is Auto Resolve IR or custom AzureIR?

Looking forward to your response.

Thank you
Pritpal 6 Reputation points

2020-10-01T18:47:28.807+00:00

Hi KranthiPakala,
Please see my response below for your questions

a) Queued Pipeline runID & activity runID ?
Ans: runId - 0e47d4d9-89d7-4494-984d-c6ee6f05c1c9
pipelineRunId - 8fbaccb6-0321-4faa-a01b-1c687022ddb3

b) Data Factory region
Ans: Canada Central

c) What is the integration runtime used for both Source and sink connection?
Ans: for source the its a custom IR created on our Azure VM.
For Destination, its Azure custom IR with Canada central location.
KranthiPakala-MSFT 46,512 Reputation points Microsoft Employee

2020-10-02T21:13:50.98+00:00

Hi @Pritpal ,

Thanks for your response and details. After looking at the logs for pipelineRun - 8fbaccb6-0321-4faa-a01b-1c687022ddb3, it seems that your pipeline run was successful.
Could you please confirm if you are seeing a different status in your Monitor section of ADF UI?

The activity runId - 0e47d4d9-89d7-4494-984d-c6ee6f05c1c9 belongs to a different pipeline run: 56f263a3-8cf4-4384-be9e-9547b06217da.

Are you using type conversion in your copy activity? If not, could you please try disabling type conversion property from your pipeline JSON code and rerun your pipeline to see if that helps to resolve the issue.

Please let us know how it goes..
KranthiPakala-MSFT 46,512 Reputation points Microsoft Employee

2020-10-02T21:15:21.87+00:00
Below is how you can disable the type conversion settings from your copy activity JSON Code:

"typeConversion": false, "typeConversionSettings": { "allowDataTruncation": false, "treatBooleanAsNumber": false }

Looking forward to your response.

Thank you
KranthiPakala-MSFT 46,512 Reputation points Microsoft Employee

2020-10-05T19:49:25.783+00:00

Hi @Pritpal ,

Following up to see if the above suggestion was helpful to resolve your issue? If you issue still persists please do let me know.

Looking forward to your confirmation.
Thank you
Pritpal 6 Reputation points

2020-10-06T15:54:41.797+00:00

Hi KranthiPakala, I re-ran the pipeline and its still in queue status. Below is the latest runid and the screenshot
RunId - 7e231112-c025-451e-bdbb-da40200e32d2

I also did the changes regarding type conversion that you specified. Still the same issue.
Pritpal 6 Reputation points

2020-10-06T16:06:29.657+00:00

Hi, I reran the pipeline and still in queue status. I also did changes regarding typeConversion but still the same issue..
Pritpal 6 Reputation points

2020-10-06T16:22:50.893+00:00

Can you please let me know how can I access the pipeline logs?
KranthiPakala-MSFT 46,512 Reputation points Microsoft Employee

2020-10-06T16:39:06.173+00:00

Hi @Pritpal ,

Sorry that you are still experiencing the issue. The backend logs can be accessed only by ADF team. Are you using the latest version of Integration Runtime? If not, could you please try upgrading your IR to latest version and see if that helps to resolve the issue

In case if the issue still persists, we may have to deep dive into SHIR logs. For deeper investigation and immediate assistance, If you have a support plan you may file a support ticket, else could you please send an email to AzCommunity[at]Microsoft[dot]com with the below details, so that we can enable a one-time-free support for you to work closely on this matter.

Email subject: <Attn - Kranthi : Microsoft Q&A Thread title>
Thread URL: <Microsoft Q&A Thread URL>
Subscription ID: <your subscription id>

Looking forward to your response.

Thank you
KranthiPakala-MSFT 46,512 Reputation points Microsoft Employee

2020-10-09T21:10:29.837+00:00

Hi @Pritpal ,

Following up to see if you have got a chance to try upgrading SHIR to latest version and how it goes? If the issue still persists, please do send email with the details requested in my previous comment.

Looking forward to your confirmation.

Thanks

Share via

ADF Pipeline stuck in queued status

Your answer