Azure Copy Data Activity getting timed out

Question

Azure Copy Data Activity getting timed out

Rajas Thakur 61

Hello,

We are getting a timeout error in a Copy Data Activity. Please see the screenshot below -

The activity runs for 2 hours (which is the timeout limit) and then times out. It seems that the rows are not getting read from the source.
In some instances, this activity get completed successfully. But it times out in some instances. Please refer the screenshot below of a successful run.

Any possible resolution to this issue?

Answer accepted by question author

0 additional answers

Your answer

Answer 1

KranthiPakala-MSFT 46,737 Microsoft Employee Moderator

Hello @Rajas Thakur ,

Thanks for the question and using MS Q&A platform.

Seems like you are trying to copy data from ADLS Gen2 to ADLS Gen2 (both in South Central US) but it is taking more than 2 hours and timing out. Please correct if I'm wrong.

I see that you are seeing using Azure IR for both Source and sink. Is it auto resolve IR or Custom Azure IR specific to South Central US? If it is Default Azure IR (Auto Resolve) the, could you please try below workarounds and see if you notice a change in the behavior.

If you are currently using Auto-Resolve Azure IR (Default), then please try creating an Azure IR specific to South central US IR,
If Option 1 didn't work, then please try creating an Azure IR pointing to a different location other than South Central US IR.
If option 2 doesn't work, then please use Self Hosted IR and see if that helps overcome this issue.
If neither option works, then I would recommend going through this troubleshooting guide to track the where actually the problem is. Troubleshoot copy activity performance

I suspect the issue could be specific to Azure IR and hence you are seeing this behavior. By chance are you using Managed Vnet IR?

Hope this info helps. Do let us know how it goes

Thank you

KranthiPakala-MSFT 46,737 Reputation points Microsoft Employee Moderator

2022-08-08T02:04:23.59+00:00

Hello @Rajas Thakur ,

Just checking in to see if the above suggestion was helped. And, if you have any further query do let us know.
Rajas Thakur 61 Reputation points

2022-08-11T19:55:39.347+00:00

Hi @KranthiPakala-MSFT ,

Thanks for the suggested solutions.

To answer your questions -
We are using a default Azure IR and we are not using a managed Vnet IR.

So I have created a new IR located in South Central US and have been using that IR in the pipeline which times out. So far, we have only received the timeout error once after making that change. I will be monitoring this for a few more days and will try with the other suggested solutions mentioned by you, incase if this doesn't work out in the long term.

I wanted to know what could be the reason of this timeout issue. We have concurrent pipelines running along with the mentioned pipeline which use that IR for data flow and copy data activities. Is that causing this issue?
Because when we run only the single pipeline (which times out), it runs fine but when we run the whole pipeline, we see that it times out in some instances.

Thanks,
Rajas
KranthiPakala-MSFT 46,737 Reputation points Microsoft Employee Moderator

2022-08-11T21:50:54.153+00:00
Hello @Rajas Thakur ,

Thanks for getting back to me and glad to know the above suggestions were helpful. And yes, since you are using the same Azure IR across multiple activities concurrently, I suspect that could be the possible reason for timeouts. I would suggest using a different Azure IR for your dataflows.

Hope this info helps.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
Rajas Thakur 61 Reputation points

2022-08-17T16:24:08.773+00:00
Hi @KranthiPakala-MSFT ,

I wanted to let you know that the option 2 worked best for me. After making that change, we haven't received the timeout error again.

To summarize, these were the steps I took to resolve this issue -

Created a new Azure IR and assigned it to the Linked Service responsible for the timeout activity.

The region of the new IR was different than the default region our data factory is in.

Thanks for the help.

Regards,
Rajas Thakur
KranthiPakala-MSFT 46,737 Reputation points Microsoft Employee Moderator

2022-08-17T17:09:58.603+00:00

Hello @Rajas Thakur ,

Appreciate much for updating the thread as it would be helpful for others reading this thread.

Thank you

Share via

Azure Copy Data Activity getting timed out

0 additional answers

Your answer