Okta System Logs connector limitations

Important

This feature is in Beta. To use it, a workspace admin must turn on Lakeflow Connect for Okta System Logs from the Previews page. See Manage Azure Databricks previews.

The managed Okta System Logs connector ingests only the system_logs table. Okta retains System Log data for 90 days, which limits how far back the connector can ingest events.

General software as a service (SaaS) connector limitations

The limitations in this section apply to all SaaS connectors in Lakeflow Connect.

• When you run a scheduled pipeline, alerts don't trigger immediately. Instead, they trigger when the next update runs.
• When a source table is deleted, the destination table is not automatically deleted. You must delete the destination table manually. This behavior is not consistent with Spark Declarative Pipelines on Lakeflow behavior.
• During source maintenance periods, Databricks might not be able to access your data.
• If a source table name conflicts with an existing destination table name, the pipeline update fails.
• Multi-destination pipeline support is API-only.
• You can optionally rename a table that you ingest. If you rename a table in your pipeline, it becomes an API-only pipeline, and you can no longer edit the pipeline in the UI.
• If you select a column after a pipeline has already started, the connector does not automatically backfill data for the new column. To ingest historical data, manually run a full refresh on the table.
• Databricks can't ingest two or more tables with the same name in the same pipeline, even if they come from different source schemas.
• The source system assumes that the cursor columns are monotonically increasing.
• The connector ingests raw data without transformations. Use downstream Spark Declarative Pipelines on Lakeflow pipelines for transformations.

Connector-specific limitations

The following limitations apply to the Okta System Logs connector:

• The connector ingests only the system_logs table. It doesn't ingest other Okta API resources, such as users, groups, applications, or policies.
• Okta retains System Log data for up to 90 days. The connector can't ingest events older than this retention window.
• The connector supports SSWS API token authentication only.
• SCD Type 2 isn't supported because System Log events are append-only.
• Row filtering isn't supported.