Upsert in copy activity taking long time.

NAGA SANDEEP KUMAR KAPA 51 Reputation points
2022-09-05T06:24:03.347+00:00

Hi Team,

I have a pipeline in which we are using copy activity with upsert functionality with sink as Azure SQL DB , it is running dead slow. It is working well in dev environment it took only 2.5 mins for 70k records whereas in acceptance it is about 191k records it took 2 hours for 30k records before i cancelled . Nothing has changed in the sink , same configuration.

How can I know where is the bottleneck. I can see in detail section writing to sink is taking long time. But how to reduce the timing.

I have tried removing index it is working but upsert functionality is not working and also I should not change the sink. How can I resolve this. Thanks in advance

Thanks,
Sandeep

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,487 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Syed Ali 5 Reputation points
    2023-06-05T21:33:21.4433333+00:00

    Could someone please advise. I have same issue and it is taking too long.

    1 person found this answer helpful.
    0 comments No comments