Share via

Copy Dataverse data into Azure SQL Error

Anonymous
2022-10-20T07:14:29.577+00:00

Hi,

I am trying to set up a pipeline to copy data using Azure Synapse link to Azure SQL database using the template 'Copy Dataverse data into azure SQL"
https://learn.microsoft.com/en-us/power-apps/maker/data-platform/azure-synapse-link-pipelines?tabs=synapse-analytics

I am getting following error on the lookup activity on model.json
"The size 5789722 of lookup activity result exceeds the limitation 4194304".

I have checked the size of the model.json file and it is only 2.8 MB. I cannot understand why this error is coming up. Any suggestion on how can this be rectified?

252306-image.png

Azure Synapse Analytics
Azure Synapse Analytics

An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Samy Abdul 3,376 Reputation points
    2022-10-20T08:18:45.393+00:00

    Hi @Anonymous , the error is more to do with lookup activity limitations which is

    The Lookup activity can return up to 5000 rows; if the result set contains more records, the first 5000 rows will be returned.

    The Lookup activity output supports up to 4 MB in size, activity will fail if the size exceeds the limit.

    The workaround is to break the source file in to smaller chunks. Thanks

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.