Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Delta Lake on Azure Databricks supports two isolation levels that control how concurrent operations on a given table interact:
| Isolation level | Description |
|---|---|
| Serializable | The strongest isolation level. Ensures that committed write operations and all reads are Serializable. Operations are allowed as long as there is a serial sequence that, when executed one at a time, generates the same outcome as seen in the table. For write operations, this serial sequence is the same as the order seen in the table's history. |
| WriteSerializable (Default) | A weaker isolation level than Serializable. Ensures that only write operations (not reads) are serializable. This is still stronger than Snapshot isolation. Provides a good balance of data consistency and availability for most common operations. |
How isolation levels affect reads
Read operations always use snapshot isolation. The write isolation level determines whether a reader can see a snapshot of a table that, according to the history, "never existed."
- Serializable: A reader always sees only tables that conform to the history
- WriteSerializable: A reader can see a table state that does not exist in the Delta log
Example: Concurrent delete and insert
Consider a scenario where a long-running delete transaction and an insert transaction start at the same time and read version v0. The insert transaction commits first and creates version v1. After that, the delete transaction attempts to commit v2:
t0: deleteTxn_START
t1: insertTxn_START
t2: insertTxn_COMMIT(v1)
t3: deleteTxn_COMMIT(v2)
In this scenario, deleteTxn did not see the data inserted by insertTxn and did not delete them:
- Serializable:
deleteTxnis not allowed to commit, and a conflict occurs - WriteSerializable:
deleteTxnis allowed to commit because the transactions can be ordered. The resulting table state is as ifinsertTxnoccurred afterdeleteTxn, so the inserted rows are part of the table. However, the Delta history shows the physical commit order (insertTxnat v1 beforedeleteTxnat v2).
Set the isolation level
Set the isolation level using the ALTER TABLE command:
ALTER TABLE <table-name> SET TBLPROPERTIES ('delta.isolationLevel' = <level-name>)
Where <level-name> is Serializable or WriteSerializable.
Example:
-- Change from default WriteSerializable to Serializable
ALTER TABLE my_table SET TBLPROPERTIES ('delta.isolationLevel' = 'Serializable')
When does Delta Lake commit without reading the table?
Delta Lake INSERT or append operations do not read the table state before committing if the following conditions are satisfied:
- Logic is expressed using
INSERTSQL logic or append mode - Logic contains no subqueries or conditionals that reference the table targeted by the write operation
As with other commits, Delta Lake uses transaction log metadata to validate and resolve table versions on commit, but no version of the table is actually read.
Note
Many common patterns use MERGE operations to insert data based on table conditions. Although it might be possible to rewrite this logic using INSERT statements, if any conditional expression references a column in the target table, these statements have the same concurrency limitations as MERGE.