ADF pipeline failing with 'unable to acquire 65536 bytes of memory'

Question

ADF pipeline failing with 'unable to acquire 65536 bytes of memory'

Rai, Vrinda 21

Hi,

We are trying to copy file of around 8 GB from Azure data lake storage account to another folder on data lake after some processing. Pipeline is failing in Data Flow with below error message -

"Operation on target Schedule_SAP_Purchase failed: Operation on target LoopToIterateOverQaulifiedSourceTables failed: Activity failed because an inner activity failed; Inner activity name: GenericBusinessNameConversion, Error: Operation on target GenericBusinessNameConversion_withPartition failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: null\n\nPrevious exception in task: Unable to acquire 65536 bytes of memory, got 0\n\torg.apache.spark.memory.MemoryConsumer.throwOom(MemoryConsumer.java:157)\n\torg.apache.spark.memory.MemoryConsumer.allocateArray(MemoryConsumer.java:98)\n\torg.apache.spark.util.collection.unsafe.sort"

Please help in debugging the issue.

Thanks !!

Accepted answer

1 additional answer

Your answer

Answer 1

Hi @Rai, Vrinda ,

Welcome to Microsoft Q&A forum and thanks for reaching out.

By looking at the error message seems like it is related to Out of memory issue. May be cluster ran into out of memory issue during execution of your transformation in mapping data flow. Hence please retry using an integration runtime with bigger core count and/or memory optimized compute type. Using appropriate compute type and size to run the payload might help to avoid this issue.

Another reason could user configuration issue. This is with respect to your logic/code in the transformation. I would also recommend you to please check your transformation's logics to see if anything is causing this memory issue.

If you still experience the issue even after applying mitigation steps, then I would recommend you to please file a support ticket so that a support engineer can take a deeper look into your pipeline transformation and identify the root cause.

Hope this info helps.

----------

Please don't forget to click on and upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Rai, Vrinda 21 Reputation points

2021-11-18T06:30:11.907+00:00

Hi Kranthi,

Thanks for your response !
We are passing Core Count through query output to pipeline activity and we already tried changing Core value to 16, 64 and 256 values but still getting same error. File is around 8 GB size. We also tried using memory optimized compute type but it also did not help.
Can you please let us know how to raise support ticket to Microsoft for this issue.

Thanks,
Vrinda
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2021-11-18T06:55:24.463+00:00

Thanks for getting back @Rai, Vrinda . If you have a support plan, you can follow this document to open a support ticket - Create an Azure support request
Please do share the support request number once it is filed so that we can track it internally.

In case if you don't have a support plan, please let us know here.

Thanks

Answer 2

I am also getting same issue can some please help. I have tried event based trigger also but same issue.

My Issue is here - https://learn.microsoft.com/en-us/answers/questions/742024/adf-job-failed-due-to-reason-cannot-grow-bufferhol.html
Details Related To My Issue - https://twitter.com/ImAbhishekTomar/status/1494711465126682627?s=20&t=_75f0dMV8kvFQCSU2JBsNQ

My Issue Is Same Like This -https://stackoverflow.com/questions/69896806/df-executor-outofmemoryerror-in-synapse-pipeline

Share via

ADF pipeline failing with 'unable to acquire 65536 bytes of memory'

1 additional answer

Your answer