Edit

Mirrored Google Lakehouse runtime catalog

A mirrored Google Lakehouse runtime catalog in Fabric integrates the Apache Iceberg tables managed in your Google Cloud Lakehouse with the rest of your data in Fabric. You can continuously mirror your Google Lakehouse runtime catalog structure directly into Fabric's OneLake.

For a tutorial on configuring your Google Cloud environment for catalog mirroring into Fabric, see Tutorial: Configure mirrored Google Lakehouse runtime catalog.

Important

This feature is in preview.

When you mirror a Google Lakehouse runtime catalog, there's no data movement or duplication. Only the catalog structure is mirrored to Fabric. The underlying Iceberg table data stays in Google Cloud Storage and is accessed through shortcuts. Changes to the underlying data might not appear in Fabric immediately. Propagation can take anywhere from a few seconds to several minutes, as described in the SQL analytics endpoint performance documentation.

What analytics experiences are built in

Mirrored catalogs are items in Fabric distinct from other items such as lakehouses and SQL analytics endpoints.

When you mirror a Google Lakehouse runtime catalog, Fabric creates these items:

You can access your mirrored Google Lakehouse runtime catalog data in multiple ways:

  • Use T-SQL commands to define and query data objects from the read-only SQL analytics endpoint. Each mirrored Google Lakehouse runtime catalog item has an autogenerated SQL analytics endpoint that provides a rich analytical experience created by the mirroring process.
  • Use Power BI with Direct Lake mode to create reports against the mirrored catalog item.
  • Use other Fabric workloads to query tables in the mirrored catalog item.

Mirrored tables are available to query within seconds after selection, with end-to-end metadata propagation typically completing in seconds to a few minutes.

Connect and authenticate

Mirroring connects to the Apache Iceberg REST catalog endpoint in the Google Lakehouse runtime catalog. The underlying table data stays in Google Cloud Storage and is read through shortcuts, so the connection needs permission to read both the catalog metadata and the table data.

Google Lakehouse runtime catalog mirroring uses Google Cloud Workload Identity Federation to authenticate Microsoft Entra identities. Fabric presents a Microsoft Entra OpenID Connect (OIDC) token to Google Cloud instead of storing a Google Cloud service account key.

Before creating the Fabric connection, configure a Google Cloud Workload Identity Pool with an OIDC provider that trusts Microsoft Entra tokens. Use these exact provider values:

Provider setting Value
Provider type OpenID Connect (OIDC)
Issuer URL https://sts.windows.net/{TENANT_ID}/
Allowed audience https://analysis.windows.net/powerbi/connector/MirroredGoogleLakehouseRuntimeCatalog
Attribute mapping google.subject = assertion.oid

Add each Microsoft Entra identity that needs access as a federated IAM principal, or authorize a principal set for the pool. Individual identity bindings are recommended for least-privilege access. Grant the principal the BigLake Viewer role (roles/biglake.viewer) and the Service Usage Consumer role (roles/serviceusage.serviceUsageConsumer) at the narrowest appropriate resource scope.

Record the numeric Google Cloud project number, Workload Identity Pool ID, and OIDC provider ID. You need these values to create the Fabric connection.

The source catalog must also be able to authorize access to its Cloud Storage data. For a catalog that uses credential vending, grant the catalog's auto-provisioned service account the Storage Object User role (roles/storage.objectUser) on all associated Cloud Storage buckets. For a catalog that uses end-user credentials, grant the federated principal the required read access to the associated buckets.

For step-by-step instructions to configure Workload Identity Federation and the connection, see Tutorial: Configure mirrored Google Lakehouse runtime catalog.

Metadata sync

When you create a mirrored Google Lakehouse runtime catalog in Fabric, the Automatically sync future tables option is enabled by default. If you enable automatic sync, your Google Lakehouse runtime catalog automatically reflects the following metadata changes in Fabric:

  • Addition of namespaces to the catalog
  • Deletion of namespaces from the catalog
  • Addition of tables to a namespace
  • Deletion of tables from a namespace
  • When Automatically sync future tables is enabled, newly created Iceberg tables in selected namespaces are automatically included in Fabric without requiring a manual refresh.
  • Schema and data changes to existing tables follow normal propagation intervals.

Namespace and table selection works as follows:

  • You can select which namespaces to include as schemas, and you can exclude specific tables.
  • Unselecting a namespace unselects all the tables within the namespace.
  • If you select the namespace again, all tables within the namespace are selected again.

Related content

  • Last updated on