Isolation levels (WriteSerializable and Serializable)

Delta Lake on Azure Databricks supports two isolation levels that control how concurrent operations on a given table interact:

Isolation level Description
Serializable The strongest isolation level. Ensures that committed write operations and all reads are Serializable. Operations are allowed as long as there is a serial sequence that, when executed one at a time, generates the same outcome as seen in the table. For write operations, this serial sequence is the same as the order seen in the table's history.
WriteSerializable (Default) A weaker isolation level than Serializable. Ensures that only write operations (not reads) are serializable. This is still stronger than Snapshot isolation. Provides a good balance of data consistency and availability for most common operations.

When to change the isolation level

Keep the default WriteSerializable isolation level for most workloads.

With WriteSerializable, a DELETE, UPDATE, or MERGE that starts before a concurrent insert can still commit after it. The rows from the insert aren't affected, even if they match the operation's condition. The table history still lists the insert first, so the table contents don't match the result of replaying the commits in history order. See Example: Concurrent delete and insert.

Change a table to Serializable if your workload can't accept this outcome. For example, if a DELETE must remove every matching row that exists when it commits, or if downstream processes reconstruct table contents from the commit order in the history. With Serializable, the later operation fails with a conflict instead, and you must retry it.

Serializable reduces throughput for concurrent writes. Appends that don't read the table never conflict under WriteSerializable but can conflict under Serializable. See Row-level concurrency.

How isolation levels affect reads

Read operations always use snapshot isolation. The write isolation level determines whether a reader can see a snapshot of a table that, according to the history, "never existed."

  • Serializable: A reader always sees only tables that conform to the history
  • WriteSerializable: A reader can see a table state that does not exist in the Delta log

Example: Concurrent delete and insert

Consider a scenario where a long-running delete transaction and an insert transaction start at the same time and read version v0. The insert transaction commits first and creates version v1. After that, the delete transaction attempts to commit v2:

t0: deleteTxn_START
t1: insertTxn_START
t2: insertTxn_COMMIT(v1)
t3: deleteTxn_COMMIT(v2)

In this scenario, deleteTxn did not see the data inserted by insertTxn and did not delete them:

  • Serializable: deleteTxn is not allowed to commit, and a conflict occurs
  • WriteSerializable: deleteTxn is allowed to commit because the transactions can be ordered. The resulting table state is as if insertTxn occurred after deleteTxn, so the inserted rows are part of the table. However, the Delta history shows the physical commit order (insertTxn at v1 before deleteTxn at v2).

Set the isolation level

To set the isolation level, use the ALTER TABLE command:

ALTER TABLE <table-name> SET TBLPROPERTIES ('delta.isolationLevel' = <level-name>)

Where <level-name> is Serializable or WriteSerializable.

Example:

-- Change from default WriteSerializable to Serializable
ALTER TABLE my_table SET TBLPROPERTIES ('delta.isolationLevel' = 'Serializable')

When does Delta Lake commit without reading the table?

Delta Lake INSERT or append operations do not read the table state before committing if these conditions are met:

  1. Logic is expressed using INSERT SQL logic or append mode
  2. Logic contains no subqueries or conditionals that reference the table targeted by the write operation

As with other commits, Delta Lake uses transaction log metadata to validate and resolve table versions on commit, but no version of the table is actually read.

Note

Many common patterns use MERGE operations to insert data based on table conditions. Although it might be possible to rewrite this logic using INSERT statements, if any conditional expression references a column in the target table, these statements have the same concurrency limitations as MERGE.

Additional resources