Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,624 questions
Delta version upgrade in ADF
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 55.00000000000001%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVP%3C/text%3E%3C/svg%3E)
Venkateswarlu Pottapalli
40
Reputation points Microsoft External Staff
Hi,
I am facing the issue to read file in Blob (delta file) using dataflows in ADF.
you can see the error message below.
**"**Job failed due to reason: at Source 'ecadl': Delta table "XXXXXXXXXXXXXXXXXX" with minReaderVersion of 3 and minWriterVersion of 7 is not supported by this version of Delta Lake 2.2 with reader version of 2 and writer version of 5. Please upgrade to newer Delta Lake version."
Based on the error I understood that delta version to be upgraded in ADF. So could you please help me how can we upgrade.
Thanks in Advance,
Venkat
Azure Data Factory
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 99%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJN%3C/text%3E%3C/svg%3E)
J N S S Kasyap 3,625 Reputation points Microsoft External Staff Moderator2025-03-17T20:16:11.4233333+00:00 Hi @Venkateswarlu Pottapalli
ADF's Mapping Data Flows do not support Delta Lake tables with higher protocol versions, such as those requiring reader version 3 and writer version 7. This limitation means that ADF cannot directly read or write Delta tables that utilize features introduced in these higher protocol versionsRecommended solutions:
Use Azure Databricks for Delta Processing:
Azure Databricks provides up-to-date support for the latest Delta Lake features and protocol versions, ensuring compatibility with your Delta tables.
To integrate Azure Databricks with Azure Data Factory (ADF) for processing Delta tables, set up a Databricks cluster with the required Delta Lake version. Then, create a Databricks notebook for the necessary data transformations. Finally, use ADF's Azure Databricks activity to trigger the notebook within your ADF pipelines.Convert Delta to Parquet
To process Delta tables using Azure Data Factory (ADF) and avoid version compatibility issues, you can read the Delta table in Databricks, write the data as Parquet files to a storage location accessible to ADF, and then configure ADF to read and process the Parquet files as needed.
Downgrade Delta Table Protocol Version
Aligning your Delta tables to a protocol version compatible with ADF's current capabilities can eliminate the version mismatch.
Implementation:- Assess whether your Delta tables use features exclusive to higher protocol versions.
- Use Databricks to modify the Delta table's properties, setting
minReaderVersionandminWriterVersionto versions supported by ADF.
ALTER TABLE <table identifier> SET TBLPROPERTIES(‘delta.minReaderVersion’ = ‘1’ , ‘delta.minWriterVersion’ = ‘3’);I hope this information helps. Please do let us know if you have any further queries.
Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.
Thank you.
-
J N S S Kasyap 3,625 Reputation points Microsoft External Staff Moderator
2025-03-18T16:29:04.6133333+00:00 @Venkateswarlu Pottapalli
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help. -
J N S S Kasyap 3,625 Reputation points Microsoft External Staff Moderator
2025-03-19T16:16:27.89+00:00 @Venkateswarlu Pottapalli
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Sign in to comment
Sign in to answer