Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Important
This feature is in Beta. Workspace admins can control access to this feature from the Previews page by opting into the Lakeflow Connect for OneDrive For Business preview. See Manage Azure Databricks previews.
The managed OneDrive connector in Lakeflow Connect allows you to ingest files from a user's OneDrive for Business My files drive into Azure Databricks. Ingest unstructured files as binary data or managed FILE values, parse structured formats (CSV, JSON, XML, EXCEL, and more) into Delta tables, or capture file metadata without loading file contents.
To ingest files from SharePoint sites, team sites, or shared document libraries, use the managed SharePoint connector.
To read from OneDrive using SQL and Spark APIs (read_files, spark.read, Auto Loader), see Ingest files from OneDrive.
What to know before you start
| Topic | Why it matters |
|---|---|
| Connection privileges | To create a connection, you need the CREATE CONNECTION privilege on the metastore. To create a pipeline from an existing connection, you need the USE CONNECTION privilege or ALL PRIVILEGES on the connection. |
| Authentication method | The steps to create a connection depend on the authentication method you select. |
| Interface | The steps to create a pipeline depend on the interface. UI-based pipeline authoring is not supported. |
| Ingestion frequency | The pipeline schedule depends on your latency and cost requirements. |
| Common patterns | Depending on your ingestion needs, the pipeline might use configurations like history tracking, column selection, and row filtering. Supported configurations vary by connector. See Feature availability. |
Start ingesting from OneDrive
The following table has an overview of the end-to-end OneDrive ingestion flow, based on connection privileges:
| Required privilege | Steps |
|---|---|
CREATE CONNECTION on the metastore |
|
USE CONNECTION or ALL PRIVILEGES on an existing connection |
Use any supported interface to create a pipeline from an existing connection. See Ingest data from OneDrive. |
Feature availability
| Feature | Availability |
|---|---|
| UI-based pipeline authoring |  Not supported |
| API-based pipeline authoring |  Supported |
| Declarative Automation Bundles | Not supported |
| Incremental ingestion | Supported |
| Unity Catalog governance | Supported |
| Orchestration using Databricks Workflows | Supported |
| SCD type 2 | Not supported |
| Schema evolution | Supported Configurable via schema_evolution_mode. See Microsoft OneDrive connector reference. |
| API-based row filtering | Supported |
Supported file formats
The connector supports both unstructured and structured file ingestion:
- Unstructured: managed file references (
FILE, Beta) andBINARYFILE. - Structured and semi-structured:
CSV,JSON,XML,EXCEL,PARQUET,AVRO, andORC.
For destination table schemas, see Ingested data format. For connector-specific limitations, see Microsoft OneDrive connector limitations.
Authentication methods
| Authentication method | Availability |
|---|---|
| OAuth U2M: Databricks-managed | Supported when enabled in the workspace No Microsoft Entra ID app registration required. If this option isn't listed, use OAuth U2M: Custom-managed. See Configure OAuth U2M: Databricks-managed for Microsoft OneDrive ingestion. |
| OAuth U2M: Custom-managed | Supported Uses your own Microsoft Entra ID app. See Configure OAuth U2M: Custom-managed for Microsoft OneDrive ingestion. |
| OAuth M2M | Not supported |
| OAuth (manual refresh token) | Not supported |
With both U2M methods, the connection authenticates as the signed-in Microsoft account, so a pipeline can ingest only the files that account can access.