Exact no of DIU calculation

Question

Hi All, I understand that a Data Integration Unit (DIU) is a representative measure of a combination of CPU, memory and network resource allocation and the default DIU setting on all copy tasks is 4, the minimum you can configure is 2 and the maximum that you can set it up is 256. How do we come to conclude on exact no of DIU needed for particular pipeline? Is any formula that determines the exact DIU to be set it up ? Thankful for your inputs.

Answer 1

There is no formula available as such to calculate DIU. Some exercise & some test runs have to be performed.. try copying small / medium / large amount of data and every time scale up the DIU number & also Parallel copies config in copy activity, see what DIU works best and note down that performance and going forward refer the DIU used from captured metrics.

Same suggestion is given in docs as well - https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-performance#performance-tuning-steps

----------

Please don't forget to Accept Answer and Up-vote if the response helped -- Vaibhav