Share via

ADF copy activity - reading from source too slow

Serban 0 Reputation points
2023-05-05T13:42:35.0133333+00:00

I have an ADF pipeline that copies tables from an on-prmemise SQL Server to Azure SQL Db. The problem is that throughput when running the copy activity will always be ~5MB/s. I tried using increased DIU, partitioning, parallelism and increasing VM size to improve the throughput. This did decrease writing to sink time (to azure sql) but not reading from source (on-premise sql). An azure integration runtime is used for this setup, together with an azure VM for network forwarding, as described here. The on premise database is able to return data much faster in SSMS and if querying from an azure function.

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. ShaikMaheer-MSFT 38,636 Reputation points Microsoft Employee Moderator
    2023-05-08T09:10:31.0233333+00:00

    Hi Serban,

    Thank you for posting query in Microsoft Q&A Platform.

    I feel using Self hosted IR from on-prem network helps in this case. Kindly try that option.

    Hope this helps. Please let me know how it goes.

    Please consider hitting Accept Answer button. Accepted answers help community as well.

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.