What's new and planned for OneLake in Microsoft Fabric
Important
The release plans describe functionality that may or may not have been released yet. The delivery timelines and projected functionality may change or may not ship. Refer to Microsoft policy for more information.
OneLake is a single, unified, logical data lake for your whole organization. Like OneDrive, OneLake comes automatically with every Microsoft Fabric tenant and is designed to be the single place for all your analytics data.
Any data in OneLake works with out-of-the-box governance such as data lineage, data protection, certification, catalog integration, etc. and is ultimately under the control of a tenant admin. Within a tenant, workspaces enable different parts of the organization to work independently while still contributing to the same data lake.
OneLake is open at every level. OneLake supports the same ADLS Gen2 APIs and SDKs to be compatible with existing ADLS Gen2 applications and can support any type of file, structured or unstructured.
OneLake aims to give you the most value possible out of a single copy of data. With OneLake shortcuts, you can unify your data across domains, clouds, and accounts by creating a reference to data stored in other file locations such as other OneLake locations or ADLS or S3 without data movement or duplication. You can also use the same data across multiple analytical engines because Fabric engines store all tabular data in the open parquet formats.There's no longer a need to copy data just to use it with another engine.
To learn more, see the documentation.
Investment areas
| Feature | Estimated release timeline |
|---|---|
| OneLake table shortcuts to Iceberg data | Q3 2024 |
| OneLake data access roles general availability | Q4 2024 |
| OneLake security model | Q1 2025 |
| OneLake shortcuts to on-premises data | Shipped (Q2 2024) |
| Shortcuts Google cloud storage | Shipped (Q1 2024) |
| Shortcuts API | Shipped (Q1 2024) |
| Smart caching for Amazon S3 shortcuts | Shipped (Q4 2023) |
OneLake table shortcuts to Iceberg data
Estimated release timeline: Q3 2024
Release Type: Public preview
With Microsoft OneLake, you will be able to create table shortcuts to Iceberg data in other locations, and this data will appear automatically with the Delta Lake format.
OneLake automatically produces virtualized Delta Lake metadata and makes it available through the OneLake shortcut. This means you can use your Iceberg data with all Fabric workloads that work with Delta Lake formatted data.
OneLake data access roles general availability
Estimated release timeline: Q4 2024
Release Type: General availability
OneLake data access roles build upon the existing capabilities of OneLake’s security model to increase the granularity at which security can be applied within a Fabric data item. This feature adds an inheritable RBAC (role-based access control) model that simplifies user and permissions management for data in OneLake. You can define security roles that grant read access to specific folders in OneLake, and assign them to users or groups. The access permissions determine what folders users see when accessing the lake view of the data, either through the lakehouse UX, notebooks, or OneLake APIs.
OneLake security model
Estimated release timeline: Q1 2025
Release Type: Public preview
Managing data security across multiple analytical engines and copies of data is challenging. OneLake and Fabric simplify this by enabling the use of a single data copy across multiple analytical engines without any data movement or duplication. Taking the "one copy" concept further, OneLake is also enhancing security with a finer-grain model, allowing for table and folder access in addition to row and column level security. These security definitions live with the data and travel across shortcuts to wherever the data is used. Security defined at OneLake is universally enforced no matter which analytical engine is used to access the data.
Shipped feature(s)
OneLake shortcuts to on-premises data
Shipped (Q2 2024)
Release Type: Public preview
Microsoft OneLake shortcuts is expanding to include on-premises and network-restricted data sources. With this capability, you can unify your on-prem and cloud data in OneLake.
During creation of shortcuts to AWS S3, Google Cloud Storage, or S3 compatible buckets, you will be able to optionally select an on-premises data gateway (OPDG) to establish connectivity.
Shortcuts Google cloud storage
Shipped (Q1 2024)
Release Type: Public preview
OneLake will expand shortcut support to Google Cloud Storage, allowing virtualization of data without moving or duplicating it. This enables the integration of Google Cloud Storage data with Microsoft services like ADLS gen2, OneLake, Dataverse, and Amazon S3. The data will appear and work as if it was in OneLake and gives you a simple data lake that can span clouds.
Shortcuts API
Shipped (Q1 2024)
Release Type: Public preview
A public REST API to automate creation and management of shortcut scenarios.
Smart caching for Amazon S3 shortcuts
Shipped (Q4 2023)
Release Type: Public preview
Smart caching for Amazon S3 shortcuts reduces egress costs and enhances performance by bringing data closer to the compute engine. Smart caching will egress data from S3 once, cache it locally in OneLake for a certain period. This way it eliminates the need for repeated data retrieval from S3. The cached data can be reused across multiple users, analytical engines, and scenarios that optimize the value a single egress.