Microsoft Entra ID connector limitations

Important

This feature is in Beta. To use it, a workspace admin must turn on Lakeflow Connect for Microsoft Entra ID from the Previews page. See Manage Azure Databricks previews.

This page contains information about known limitations of the managed Microsoft Entra ID connector in Lakeflow Connect.

General software as a service (SaaS) connector limitations

The limitations in this section apply to all SaaS connectors in Lakeflow Connect.

• When you run a scheduled pipeline, alerts don't trigger immediately. Instead, they trigger when the next update runs.
• When a source table is deleted, the destination table is not automatically deleted. You must delete the destination table manually. This behavior is not consistent with Spark Declarative Pipelines on Lakeflow behavior.
• During source maintenance periods, Databricks might not be able to access your data.
• If a source table name conflicts with an existing destination table name, the pipeline update fails.
• Multi-destination pipeline support is API-only.
• You can optionally rename a table that you ingest. If you rename a table in your pipeline, it becomes an API-only pipeline, and you can no longer edit the pipeline in the UI.
• If you select a column after a pipeline has already started, the connector does not automatically backfill data for the new column. To ingest historical data, manually run a full refresh on the table.
• Databricks can't ingest two or more tables with the same name in the same pipeline, even if they come from different source schemas.
• The source system assumes that the cursor columns are monotonically increasing.
• The connector ingests raw data without transformations. Use downstream Spark Declarative Pipelines on Lakeflow pipelines for transformations.

Connector-specific limitations

The limitations in this section apply to the Microsoft Entra ID connector.

• Only directoryaudits and signins support incremental ingestion. The connector refreshes all other tables in full on every pipeline update.
• On the first sync, directoryaudits and signins ingest at most the previous 30 days of events.
• The connector skips identityproviders for tenants in the us_gov_gcc_high and us_gov_dod clouds, so the table is empty for those tenants.
• The directoryroles and directory_role_member tables include only directory roles that are activated in the tenant. directory_role_member doesn't include scoped role assignments.
• The signins table contains interactive user sign-ins only. It doesn't include service principal or managed identity sign-ins.
• The group_members table contains direct members only. The connector doesn't expand nested groups.
• The group_owner table doesn't include service principal owners. It has no rows for Exchange-created, distribution, or on-premises-synced groups.
• The user_registration_detail table covers enabled users only.
• Without CustomSecAttributeAssignment.Read.All, the customSecurityAttributes column is empty. Without Policy.Read.All, the appliedConditionalAccessPolicies column of signins is empty.
• The connector doesn't ingest custom directory extensions, schema extensions, or open extensions.