ADF copy data activity issue

Ninos 10 Reputation points
2023-02-08T08:40:40.2333333+00:00

Hello. I have a copy data activity that is supposed to bring records from pcsoft database (odbc) into Azure sql database.

My self hosted IR has 24 go ram.

The source table contains 1m8 rows.

The copy activity runs everyday by a trigger, the problem is that it shows success with 0 row copied.

The issue hits randomly, sometimes I just rerun manually my pipeline and it goes fine.

This morning I tried to rerun manually my pipeline 3 times but its not working.

I've been working on this intermittently for months now and it's getting very frustrating.

Can someone help please ?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,628 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. BhargavaGunnam-MSFT 26,331 Reputation points Microsoft Employee
    2023-02-13T23:20:19.1233333+00:00

    Hello @Ninos ,

    Welcome to the MS Q&A platform.

    From your description, it could be an issue with resource consumption. Could you please confirm whether the self-hosted IR has enough memory and CPU to process the copy activity?

    and please check any issues with the network between the SHIR and PCSoft database and there are no firewall or network security group rules that are blocking the connection

    also, can you please check if there are any blocking/high resource utilization/throttling errors on the Azure SQL end when copying data using the copy activity?

    It would be great if you could share any error messages for further troubleshooting of your issue.

    0 comments