Performance issue in copy data activity with low throughput

Question

Hi All,

The problem is that the copy activity is very slow. It take about more than 3 hours to read 15GB and write 15 GB data.
The throughput is maximum 1.967 MB/s.
My configuration:
DIU units (in copy activity setting): 4
Used Parallel copies :1
source and sink are in same region
Service Tier: Standard(For workloads and typical performance requirements).

Can Anyone please help me with this how to handle this.

Please note past 2 days onwards facing this slowness with less through put before its completed successfully with in 50 mins with the through put maximum of 4.527 MB/s

Answer 1

Hi @Raj0125 , you could please looked in to these common measures to resolve

the performance, such as :

Increasing DIU
Increasing degree parallelism
Increasing concurrency of pipeline
Splitting single file (smaller files)
Set max rows per file

Also, below threads should be quite useful. Thanks

https://learn.microsoft.com/en-us/answers/questions/180644/how-to-optimize-azure-data-factorypipeline.htm

https://learn.microsoft.com/en-us/answers/questions/36323/adf-performance-troubleshooting.htm

https://github.com/Huachao/azure-content/blob/master/articles/data-factory/data-factory-copy-activity-performance.md

https://stackoverflow.com/questions/70531362/how-to-increase-performance-of-azure-data-factory-pipeline