Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,358 questions
While trying to copy excel file as parquet using get metadata and copy data activity i got the below exception. i have multiple excel which need to be copy as parquet for each file and each excel file is 70MB
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(25.6, 81%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAA%3C/text%3E%3C/svg%3E)
Avinash Agrawal (AC Kolkata)
0
Reputation points
While trying to copy excel file as parquet using get metadata and copy data activity i got the below exception. i have multiple excel which need to be copy as parquet for each file and each excel file is 70MB ErrorCode=SystemErrorOutOfMemory,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=A task failed with out of memory.,Source=,''Type=System.OutOfMemoryException,Message=Exception of type 'System.OutOfMemoryException' was thrown.,Source=Microsoft.DataTransfer.Common,'
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 38%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EQ%3C/text%3E%3C/svg%3E)
QuantumCache 20,346 Reputation points2024-02-08T00:57:05.5066667+00:00 Hello @Avinash Agrawal (AC Kolkata)Thanks for reaching out on this forum. Could you please share more details on the pipeline? is it using the SHIR ? What is the VM size? is it local dev machine? Which activity was failing, any screenshots of the failure logs? Did you consider testing the same pipeline with smaller size Excel file? I hope you have come across the below document! Parquet format in Azure Data Factory and Azure Synapse Analytics
-
QuantumCache 20,346 Reputation points
2024-02-15T18:07:16.0366667+00:00 Hello @Avinash Agrawal (AC Kolkata) Just checking if we are still connected on this discussion? Please let us know if you need to add more info so that we better assist you!
Sign in to comment
Sign in to answer