Share via

Spark delta table- Query an earlier version of the table (time travel)

Dondapati, Navin 291 Reputation points
2021-09-20T15:51:20.52+00:00

Delta Lake time travel allows you to query an older snapshot of a Delta table.

SELECT * FROM events VERSION AS OF 0

SELECT * FROM events TIMESTAMP AS OF '2019-01-29 00:37:58'

Error:
Error :
extraneous input '0' expecting {<EOF>, ';'}(line 1, pos 33)

does not synapse support such queries?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,378 questions
0 comments
Accepted answer
  1. Thomas Boersma 806 Reputation points
    2021-09-20T17:38:58.32+00:00

    Hi @Anonymous ,

    For Synapse Spark pool:
    Unfortunately, it is not possible to time travel with Delta Lake with a SQL command within the integrated notebooks in Synapse on a spark spool, but it is possible when you load the data into a dataframe with PySpark. Here is an example / tutorial video on how to achieve time travelling with PySpark on Synapse.

    For Synapse Serverless pool:
    Delta Lake with Synapse is still in public preview. So unfortunately, time traveling is not supported in Synapse Serverless Pools at this moment (known issues and limitations of Delta Lake with Synapse). But on the upside you can vote for this feature on the Azure feedback site, by clicking here.

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.