Copy Activity performance extremely slow in ADF

Question

I have a ForEach activity that performs many copy activities. The performance of the copy activities varies wildly depending on which tables are used for the source and sink. The copy activities are performed in parallel. For example, the copy activity throughput for one copy was ~4 Kb/s, while a copy activity for another table was >1 Mb/s. There does not seem to be any consistency on which tables the poor performance is happening on. Can someone explain why this is happening?

Answer 1

Hi David,

The varying performance in your Azure Data Factory Copy Activities could be due to several factors.

Different data store types and locations for your source and sink can impact performance, as can network bandwidth and latency.

You could also consider scaling up the resources in ADF, if they are under-provisioned.

The structure and volume of data in your tables could also be affecting copy speeds.

Additionally, the Degree of Parallelism settings in your copy activities might need adjustment. Sometimes, reducing parallelism can actually improve performance due to less contention.

Hope this helps

If you found this answer useful, please consider marking it as 'Accepted.' This helps other users easily find and benefit from this information

Regards