Glean connector

Important

This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Azure Databricks previews.

The managed Glean connector in Lakeflow Connect ingests Glean insights and shortcuts data into Azure Databricks. Insights provide company-wide usage summaries for search, Assistant, and agents. Shortcuts are the go links defined in your Glean deployment.

Feature availability

Feature	Availability
UI-based pipeline authoring	Supported
API-based pipeline authoring	Supported
Declarative Automation Bundles	Supported
Incremental ingestion	Not supported
Unity Catalog governance	Supported
Orchestration using Databricks Workflows	Supported
API-based column selection and deselection	Supported
API-based row filtering	Not supported
SCD Type 2	Not supported The insights table has VARIANT columns, and both tables are ingested as full-refresh snapshots.
Automated schema evolution: New and deleted columns	Supported
Automated schema evolution: Data type changes	Not supported
Automated schema evolution: Column renames	Not supported Requires a full refresh.

Authentication methods

Authentication method	Availability
OAuth U2M	Not supported
OAuth M2M	Not supported
Basic authentication (username/password)	Not supported
API token	Supported A Glean Client API token, sent as an HTTP Bearer token. The token must carry both the Insights and Shortcuts scopes.

Ingest from Glean in 3 steps

Note

Before starting, review the Azure Databricks user persona, supported interfaces, ingestion frequency, and common patterns.

Start ingesting from Glean

1. Configure Glean for ingestion (Admins). Set up Glean to authenticate with Azure Databricks.
2. Create a Unity Catalog connection (Admins). Create a connection in Catalog Explorer so non-admins can create pipelines.
3. Create an ingestion pipeline (Admins or non-admins). Select any supported interface and create a pipeline from an existing connection.