Took a long time to load data from Azure SQL Database into ADLS Gen2 by using Azure Synapse Analytics

Question

Took a long time to load data from Azure SQL Database into ADLS Gen2 by using Azure Synapse Analytics

Liang Chen 1

Hi,

I tried to use the Copy Data Activity to load ~58GB data into ADLS Gen 2 by using Synapse Pipeline. It took around 1 hour to do so. However, when I used the Copy Data Activity with the same configuration in ADF Pipeline, it only took several minutes. I was wondering why since we both use the same pipeline in two scenarios. Is there any way that I could load data from Azure SQL Database in Synapse as fast as in ADF? Please help. Thank you very much.

MartinJaffer-MSFT 25,716 Reputation points

2022-04-06T21:37:24.59+00:00

Hello @Liang Chen and welcome to Microsoft Q&A.

As I understand you are experienceing a performance difference during data movement between Data Factory and Synapse Analytics when using identical set ups.

Is this a one-off occurrence, or does it happen every time?

As far as I am aware, there should be parity between these two. To find there is disparity is concerning, and I'm sure the engineers would want to look into this, should it be repeatable. For in-depth investigation and performance related issues, I would like you to open a support ticket to let us look at the specifics.
If you do not have a support plan, I can give you a 1-time support credit. Let me know and I'll send directions.
Liang Chen 1 Reputation point

2022-04-06T22:57:32.103+00:00

Hi @MartinJaffer-MSFT ,

Thank you for your reply. This occurs every time. The Synapse Pipeline somehow took much longer time to copy the data than the ADF Pipeline, with the same configuration details.

Yes, please invite the engineers to take an in-depth investigation on this because the Synapse pipeline might be in need of use often in the future. And I don't think I have a support plan on this. Please advise. Thank you very much!

Best,
Liang

MartinJaffer-MSFT 25,716 Reputation points

2022-04-06T21:37:24.59+00:00

Hello @Liang Chen and welcome to Microsoft Q&A.

As I understand you are experienceing a performance difference during data movement between Data Factory and Synapse Analytics when using identical set ups.

Is this a one-off occurrence, or does it happen every time?

As far as I am aware, there should be parity between these two. To find there is disparity is concerning, and I'm sure the engineers would want to look into this, should it be repeatable. For in-depth investigation and performance related issues, I would like you to open a support ticket to let us look at the specifics.
If you do not have a support plan, I can give you a 1-time support credit. Let me know and I'll send directions.
Liang Chen 1 Reputation point

2022-04-06T22:57:32.103+00:00

Hi @MartinJaffer-MSFT ,

Thank you for your reply. This occurs every time. The Synapse Pipeline somehow took much longer time to copy the data than the ADF Pipeline, with the same configuration details.

Yes, please invite the engineers to take an in-depth investigation on this because the Synapse pipeline might be in need of use often in the future. And I don't think I have a support plan on this. Please advise. Thank you very much!

Best,
Liang