Share via

Azure Data Factory - OutOfMemoryError

Nam Nguyen 31 Reputation points
2022-01-05T07:23:10.633+00:00

Source: Hubspot
Sink: parquet file in Azure blob storage

Even when i adjust Core count to 16 (+16 core), following error still pop up when running pipeline:

![162491-image.png]1

Pipeline run ID:
50d2a7b0-c483-449d-8bb8-5f846f8e0914

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,420 questions
Azure Face
Azure Face
An Azure service that provides artificial intelligence algorithms that detect, recognize, and analyze human faces in images.
155 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,089 questions
0 comments
Accepted answer
  1. HimanshuSinha-msft 19,381 Reputation points Microsoft Employee
    2022-01-05T21:40:06.177+00:00

    Hello @Nam Nguyen ,
    Thanks for the ask and using Microsoft Q&A platform .
    Just wanted to suggest you to try the below options and let us know if that helps .

    1. Please try out the Compute type to "Memory optimatized " and publish the changes and see if that helps .

    162624-image.png

    2.Also try to update the Azure IR to a higher core

    162642-image.png

    Please do let me know how it goes .
    Thanks
    Himanshu

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest