ADF - ODP Connector - OutOfMemory issue

Question

Hi,

Using the new ODP connector in ADF, we sometimes encounter a memory limitation issue when extracting large tables (even if we partition them into smaller pieces). According to an earlier Q&A memory issues should not be possible anymore with this connector, so this is strange.

What the cause of this error and how can we solve it?

Answer 1

Hello @Joey Sun ,

Welcome to the MS Q&A platform.

Could you please check the resource usage and concurrent-activity execution on the self-hosted IR node? If it has high memory usage and too many concurrent activity executions, please adjust the trigger time and internal time of activity runs.

If you can restrict too many activities execution on the same IR node simultaneously, see if it helps.

Please let me know if you have any further questions.