Reading a Delta table from ADLS Gen2 in Python

Question

Hi,

I've created a Delta table in an ADLS Gen2 and need to load the locally using Python.

I've tried using this approach unsuccessfully: https://pypi.org/project/delta-lake-reader/ with the error "Could not find transaction logs in path <> Make sure you point to the root of a delta table" though it is the root that I am pointing to.

Do you know of any other methods? Note that I tried reading the .parquet file with pd.parquet_reader without issues.

Answer 1

Hello @Johan á Rogvi-Hansen ,

Thanks for the question and using MS Q&A platform.

From the error message it clearly says as the could not find the transaction logs in the path. To resolve the issue - make sure you point to the root of a delta table" though it is the root that I am pointing to.

There are three common causes for this error message.

Cause 1: You start the Delta streaming job, but before the streaming job starts processing, the underlying data is deleted.
Cause 2: You perform updates to the Delta table, but the transaction files are not updated with the latest details.
Cause 3: You attempt multi-cluster read or update operations on the same Delta table, resulting in a cluster referring to files on a cluster that was deleted and recreated.

For more details, refer to A file referenced in the transaction log cannot be found.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is jhow you can be part of Q&A Volunteer Moderators