Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
You can bring any data accessible through Azure Databricks into a visual data prep in Lakeflow Designer.
All data ingestion in Designer starts with the Source operator. When you open a Source operator's configuration pane, you have the following options.
Browse existing tables
Click Browse existing to open the asset selector and browse tables and volumes in Unity Catalog. Use this option when your data is already in Azure Databricks.
Drag and drop a local file
Drag a CSV or Excel file from your local machine directly onto the canvas. Designer uploads the file to a volume in Unity Catalog and automatically creates a Source operator.
Note
Drag and drop of Excel files requires Excel File Format Support to be enabled. See Read and stream Excel files.
Create a table from a file
Select Create table from file in the Source configuration pane to upload a local CSV or Excel file and create a Unity Catalog table from it. Use this option when you want the data persisted as a managed table rather than stored in a volume. Additionally, for large amounts of data, this provides the best performance. Uploaded files are subject to the Unity Catalog volume file size limit. See Work with files in Unity Catalog volumes.
Import from Google Drive
To import a file from Google Drive:
- In the Source configuration pane, select Import from Google Drive.
- Select a Unity Catalog connection that has access to Google Drive.
- Provide the file URL.
This requires a Unity Catalog connection configured for Google Drive access. See Ingest files from Google Drive for how to set up connections.
Import from SharePoint
To import a file from SharePoint:
- In the Source configuration pane, select Import from SharePoint.
- Select a Unity Catalog connection that has access to your SharePoint site.
- Provide the direct file link.
Warning
To get the correct URL, right-click the file in SharePoint, select Preview, and copy the URL from the browser address bar. Do not use a sharing link.
This requires a Unity Catalog connection configured for SharePoint access. See Ingest files from SharePoint for how to set up connections.
Use Lakeflow Connect for additional sources
If a data source is not available directly in Designer, check Lakeflow Connect for a managed connector. Lakeflow Connect supports ingestion from SaaS applications (Salesforce, Workday, HubSpot, and more), databases, cloud storage, and streaming sources. After you ingest data with Lakeflow Connect into a Unity Catalog table, you can use it as a Source in Designer.