AML training_data MemoryError

Question

I am wondering if there is any proposed fix on this issue https://learn.microsoft.com/en-us/answers/questions/263618/training-auto-ml-models-from-a-python-script-via-a.html.

I am encountering exactly the same problem.

I do not think it is a memory problem. Also, everything works fine tested locally with azure core tools. But when deployed on the azure function app I get the memory error.

Answer 1

@Kyriakos Chiotis ,

Thanks for reaching out to Q&A.

The following error is seen due to memory exhaustion. I have seen issues where there were some memory leaks in the code or the memory of the VM (platform) hitting its limit.

Result: FailureException: MemoryError: Engine process terminated. This is most likely due to system running out of memory. Please retry with increased memory

Some customers run the code in their local machine where it would work fine and then deploy the function to a consumption plan app where its RAM is only 1.5 GB whereas the RAM in their local machine would be greater than 1.5 GB.

I would suggest you to run the memory profiling in the python function to analyze for potential memory bottlenecks in your functions. Please note that, Memory profiling is intended only for memory footprint analysis on development environment. Please do not apply the memory profiler on production function apps.

Memory profiling: https://learn.microsoft.com/en-us/azure/azure-functions/python-memory-profiler-reference

I hope this helps!

Please 'Accept as answer' and ‘Upvote’ if it helped so that it can help others in the community looking for help on similar topics.