CopyActivity not utilising high performance

THIMMAIAH GARI,PRASHANTH,, 201 Reputation points
2020-12-08T09:39:07.123+00:00

Hi,

we are loading data from SQL SERVER to Azure SQLDB

I have triggered 4 tables parallel, after some time 3 tables loaded and 4th table is still running which have more records and at the same time, some tables loaded quickly with high throughput. Even though it's running one table why it's not using high throughput. I am keeping auto DIU'S in copyactivity

is there any other way I can improve performance? We are using SH-IR concurrency 24.

will it increase performance if we use blob/datalake as a stage? rather directly loading from db to db?

46213-image.png

Here another activity throughput is very less.
46362-image.png

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,893 questions
{count} votes

2 answers

Sort by: Most helpful
  1. HimanshuSinha-msft 19,386 Reputation points Microsoft Employee
    2020-12-08T20:47:41.18+00:00

    Hello @THIMMAIAH GARI,PRASHANTH,, ,

    Thanks for the ask and using the forum . Since the SHIR is involoved can you please run the perf counters on the SHIR system and see how its behaves , what is the memory usages / cpu etc . This kind of isues its very difficult to resolve remorely . I am not sure if you have already gone through the read .

    One other thing which if you can test ( if possible ) is how the throughput behaves when you copy the table from SQL Azure to SQL Azure .

    Let us know as to how it goes .
    Thanks
    Himanshu


  2. THIMMAIAH GARI,PRASHANTH,, 201 Reputation points
    2020-12-18T10:45:19.327+00:00

    We found that the source system having a performance issue and we are also increasing size in IR as using same IR in multiple ADF's.