Edit

Delta Lake table format interoperability

In Microsoft Fabric, the Delta Lake table format is the standard for analytics. Delta Lake is an open-source storage layer that brings ACID (Atomicity, Consistency, Isolation, Durability) transactions to big data and analytics workloads.

All Fabric experiences natively generate and consume Delta Lake tables, providing a unified product experience. Delta Lake tables produced by one compute engine, such as Fabric Data Warehouse or Synapse Spark, can be consumed by any other engine, such as Power BI. When you ingest data into Fabric, Fabric stores it as Delta tables by default. You can easily integrate external data containing Delta Lake tables by using OneLake shortcuts.

Delta Lake features and Fabric experiences

To achieve interoperability, all the Fabric experiences align on the Delta Lake features and Fabric capabilities. Some experiences can only write to Delta Lake tables, while others can read from it.

  • Writers: Data warehouses, eventstreams, and exported Power BI semantic models into OneLake
  • Readers: SQL analytics endpoint and Power BI direct lake semantic models
  • Writers and readers: Fabric Spark runtime, dataflows, pipelines, and Kusto Query Language (KQL) databases

The following matrix shows key Delta Lake features and its availability on each Fabric experience.

Fabric capability Column mappings Deletion vectors V-order writing Table optimization and maintenance Partitions
Data warehouse Delta Lake export Name: Yes
ID: No		 Yes Yes Yes Read: N/A (not applicable)
Write: No
SQL analytics endpoint Name: Yes
ID: No		 Yes N/A (not applicable) N/A (not applicable) Read: Yes
Write: N/A (not applicable)
Lakehouse explorer and preview Name: Yes
ID: No		 Yes N/A (not applicable) Yes Read: Yes
Write: N/A (not applicable)
Fabric Spark Runtime 2.0 (Preview) Name: Yes
ID: Yes		 Yes Yes Yes Read: Yes
Write: Yes
Fabric Spark Runtime 1.3 Name: Yes
ID: Yes		 Yes Yes Yes Read: Yes
Write: Yes
Fabric Spark Runtime 1.2 Name: Yes
ID: Yes		 Yes Yes Yes Read: Yes
Write: Yes
Fabric Spark Runtime 1.1 Name: Yes
ID: Yes		 No Yes Yes Read: Yes
Write: Yes
Dataflows Gen2 Name: Yes
ID: No		 Yes Yes No Read: Yes
Write: Yes
Pipelines Name: No
ID: No		 No Yes No Read: Yes
Write: Yes, overwrite only
Power BI direct lake semantic models Name: Yes
ID: Yes		 Yes N/A (not applicable) N/A (not applicable) Read: Yes
Write: N/A (not applicable)
Export Power BI semantic models into OneLake Name: Yes
ID: No		 N/A (not applicable) Yes No Read: N/A (not applicable)
Write: No
KQL databases Name: Yes
ID: No		 Yes No No1 Read: Yes
Write: Yes
Eventstreams Name: No
ID: No		 No No No Read: N/A (not applicable)
Write: Yes

1 KQL databases provide certain table maintenance capabilities such as retention. Data is removed at the end of the retention period from OneLake. For more information, see One Logical copy.

Fabric capability Liquid Clustering TIMESTAMP_NTZ Delta reader/writer version and default table features Type Widening
Data warehouse Delta Lake export No No Reader: 3
Writer: 7
Deletion Vectors,
Column Mappings (name)		 Yes
SQL analytics endpoint Yes No N/A (not applicable) Yes
Lakehouse explorer and preview Yes Yes N/A (not applicable) Yes
Fabric Spark Runtime 2.0 (Preview) Yes Yes Reader: 3
Writer: 7
Deletion Vectors		 Yes
Fabric Spark Runtime 1.3 Yes Yes Reader: 1
Writer: 2		 No
Fabric Spark Runtime 1.2 Yes, read only Yes Reader: 1
Writer: 2		 No
Fabric Spark Runtime 1.1 Yes, read only No Reader: 1
Writer: 2		 No
Dataflows Gen2 Yes, read only No Reader: 1
Writer: 2		 No
Pipelines Yes, read only No Reader: 1
Writer: 2		 No
Power BI direct lake semantic models Yes No N/A (not applicable) No
Export Power BI semantic models into OneLake No No Reader: 2
Writer: 5
Column Mappings (name)		 No
KQL databases No No Reader: 1
Writer: 1		 No
Eventstreams No No Reader: 1
Writer: 2		 No

Note

  • Fabric doesn't write column mappings by default, except where noted. The default Fabric experience generates tables that are compatible across the service. Delta Lake tables produced by third-party services may have incompatible table features.
  • Some Fabric experiences don't offer table optimization and maintenance capabilities, such as bin-compaction, V-order, deletion vector merge (PURGE), and clean up of old unreferenced files (VACUUM). To keep Delta Lake tables optimal for analytics, follow the techniques in Use table maintenance feature to manage delta tables in Fabric for tables ingested using those experiences.
  • For comprehensive cross-workload guidance on table maintenance strategies for different consumption scenarios, see Cross-workload table maintenance and optimization.

Current limitations

Currently, Fabric doesn't support these Delta Lake features:

  • V2 Checkpoints aren't uniformly available in all experiences. Only Spark notebooks and Spark jobs can read and write to tables with V2 Checkpoints. Lakehouse and SQL analytics endpoints don't correctly list tables containing V2 Checkpoint files in the __delta_log folder.
  • Delta Lake 3.x Uniform. This feature is supported only in the Data Engineering Spark-compute (Notebooks, Spark Jobs).
  • Identity columns writing (Azure Databricks feature)
  • Lakeflow Spark Declarative Pipelines (Azure Databricks feature)
  • Delta Lake 4.x features outside of Lakehouse and Spark Notebooks and Jobs: collations, variant type, coordinated commits, etc. Type widening is supported where noted on previous features matrix in this article.

Special characters on table names

Microsoft Fabric supports special characters as part of the table names. This feature allows the usage of unicode characters to compose table names in Fabric experiences.

The following special characters are either reserved or not compatible with at least one of Fabric technologies and must not be used as part of a table name: " (double quotes), ' (single quote), #, %, +, :, ?, or ` (backtick).

Related content

  • Last updated on