Data factory parallel copy

Question

Is there anyone has experience on parallel copy from Oracle to ADLS Gen2 ?

I have tried Oracle table partition with 4 , as well as 8 , the effective parallel is just 3 (shown on the copy activity) . I am using SELF-host IR (8 cores) . Copy to ADLS Gen2 as parquet.

Answer 1

HI @sakuraime ,

Thanks for your ask. The parallel copy number what we set is just the max parallel copy to be allowed the copy activity, but during runtime the parallel copy number can be less than or equal to the number set in the copy activity. You can think of this property as the maximum number of threads within the copy activity that read from your source or write to your sink data stores in parallel. At run time, for the best performance, the copy activity uses a value that is less than or equal to the value that you set. This depends on the DIU, network, source status, sink status and other settings. For each copy activity run, by default the service dynamically applies the optimal parallel copy setting based on your source-sink pair and data pattern without exceeding the max parallel copy threads as set in the parallel copy property.

Hope this clarifies.

Thank you

----------

• Please don't forget to click on and upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators