Typical Lakebase project setup with Declarative Automation Bundles

This page shows a complete Declarative Automation Bundles bundle for a production-ready Lakebase Autoscaling project with the most commonly used features:

  • Protected production branch
  • High availability (HA) read-write endpoint with readable secondaries
  • Inline workspace-level CAN_MANAGE permission for a service principal
  • Continuous synced table streaming from Unity Catalog
  • Unity Catalog binding for the Lakebase database
  • Databricks App connected to the Lakebase project

For a step-by-step introduction to Declarative Automation Bundles with Lakebase, see Manage Lakebase with Declarative Automation Bundles.

Prerequisites

Before you begin, you need:

  • Databricks CLI v1.0.0 or later. To check your version, run databricks --version. To install or upgrade, see Install or update the Databricks CLI.
  • A Azure Databricks workspace with Lakebase enabled.
  • A service principal configured for OAuth machine-to-machine (M2M) authentication. The bundle grants this principal workspace CAN_MANAGE permission on the project. See Authorize service principal access to Azure Databricks with OAuth and Manage project permissions.
  • A Unity Catalog Delta table with Change Data Feed (CDF) enabled to use as the sync source. Remove the postgres_synced_tables and postgres_catalogs blocks if you don't need data sync.

Complete bundle configuration

The bundle uses variables for all workspace-specific values. Set them in a .databricks/bundle/<target>/variables.json file, or pass them at deploy time with --var.

When you create a project, Azure Databricks automatically creates a production branch and a primary read-write endpoint. To configure these implicitly created resources, declare them with replace_existing: true.

bundle:
  name: lakebase-typical-project

variables:
  project_id:
    description: 'Lakebase project ID (lowercase, hyphen-delimited)'
    default: 'my-lakebase-project'
  display_name:
    description: 'Human-readable project name shown in the UI'
    default: 'My Lakebase project'
  pg_version:
    description: 'Postgres major version'
    default: 17
  min_cu:
    description: 'Minimum compute units on the default endpoint'
    default: 0.5
  max_cu:
    description: 'Maximum compute units on the default endpoint'
    default: 4.0
  suspend_timeout:
    description: 'Idle time before the default endpoint suspends. Ignored when no_suspension is true.'
    default: '300s'
  admin_sp_app_id:
    description: 'Application ID of the service principal to grant CAN_MANAGE on the project'
    default: '<your-sp-application-id>'
  source_table:
    description: 'Unity Catalog three-part name of the Delta table to sync (catalog.schema.table)'
    default: '<catalog>.<schema>.<table>'
  primary_key_column:
    description: 'Primary key column of the source Delta table'
    default: '<pk>'
  storage_catalog:
    description: 'Unity Catalog catalog where the sync pipeline stores its metadata'
    default: '<catalog>'
  storage_schema:
    description: 'Unity Catalog schema where the sync pipeline stores its metadata'
    default: '<schema>'
  app_name:
    description: 'Databricks App name (must be unique in the workspace)'
    default: 'my-lakebase-app'
  uc_catalog_id:
    description: 'Name to register the Lakebase database in Unity Catalog'
    default: 'my_lakebase_uc_catalog'

targets:
  prod:
    default: true
    workspace:
      host: https://<your-workspace>.cloud.databricks.com

    resources:
      # Project — top-level container for branches, endpoints, and databases.
      # The permissions block grants workspace-level CAN_MANAGE to the service principal.
      postgres_projects:
        lakebase_project:
          project_id: ${var.project_id}
          spec:
            pg_version: ${var.pg_version}
            display_name: ${var.display_name}
            default_endpoint_settings:
              autoscaling_limit_min_cu: ${var.min_cu}
              autoscaling_limit_max_cu: ${var.max_cu}
              suspend_timeout_duration: ${var.suspend_timeout}
          permissions:
            - service_principal_name: ${var.admin_sp_app_id}
              level: CAN_MANAGE

      # Configure the implicitly created production branch as protected.
      postgres_branches:
        production:
          branch_id: production
          parent: ${resources.postgres_projects.lakebase_project.name}
          no_expiry: true
          is_protected: true
          replace_existing: true

      # Configure the implicitly created primary endpoint with HA.
      # HA requires no_suspension: true. group.min: 2 adds a standby for automatic failover.
      postgres_endpoints:
        primary:
          endpoint_id: primary
          parent: ${resources.postgres_branches.production.name}
          endpoint_type: ENDPOINT_TYPE_READ_WRITE
          autoscaling_limit_min_cu: ${var.min_cu}
          autoscaling_limit_max_cu: ${var.max_cu}
          no_suspension: true
          group:
            min: 2
            max: 2
            enable_readable_secondaries: true
          replace_existing: true

      # Sync a Unity Catalog Delta table into the project continuously.
      postgres_synced_tables:
        orders_sync:
          synced_table_id: '${var.storage_catalog}.${var.storage_schema}.orders_synced'
          branch: ${resources.postgres_branches.production.name}
          postgres_database: databricks_postgres
          source_table_full_name: ${var.source_table}
          primary_key_columns:
            - ${var.primary_key_column}
          scheduling_policy: CONTINUOUS
          create_database_objects_if_missing: true
          new_pipeline_spec:
            storage_catalog: ${var.storage_catalog}
            storage_schema: ${var.storage_schema}

      # Bind the Lakebase database into Unity Catalog so it is queryable as UC data.
      postgres_catalogs:
        lakebase_uc_catalog:
          catalog_id: ${var.uc_catalog_id}
          postgres_database: databricks_postgres
          branch: ${resources.postgres_branches.production.name}
          create_database_if_missing: true

      # Databricks App connected to the project.
      # Update source_code_path to point to your app source directory.
      # The database value references databricks_postgres — the stable ID of the implicit
      # database that every Lakebase project creates automatically.
      apps:
        lakebase_app:
          name: ${var.app_name}
          description: 'App backed by Lakebase autoscaling'
          source_code_path: ./app_src
          config:
            command:
              - flask
              - run
              - --host=0.0.0.0
              - --port=8000
          resources:
            - name: lakebase-db
              postgres:
                branch: ${resources.postgres_branches.production.name}
                database: ${resources.postgres_branches.production.name}/databases/databricks-postgres
                permission: CAN_CONNECT_AND_CREATE

Apply the bundle

Validate and deploy:

databricks bundle validate -t prod
databricks bundle deploy -t prod

If databricks bundle deploy does not complete on the first run, re-run it.

What gets deployed

The bundle creates the following resources:

  • A Lakebase Autoscaling project with the compute defaults you specified.
  • A protected production branch.
  • A primary read-write endpoint with HA and readable secondaries.
  • A continuous sync pipeline that streams a Unity Catalog Delta table into the project database.
  • A Unity Catalog catalog backed by the Lakebase database, queryable as UC data.
  • A Databricks App connected to the project database.
  • Workspace CAN_MANAGE permission for the service principal you specified.

Next steps

  • Last updated on