Breyta

Deila með

Add PostgreSQL Database CDC as source in Real-Time hub (preview)

  • Grein

This article describes how to add PostgreSQL Database Change Data Capture (CDC) as an event source in Fabric Real-Time hub.

The PostgreSQL Database Change Data Capture (CDC) source connector for Microsoft Fabric event streams allows you to capture a snapshot of the current data in a PostgreSQL database. The connector then monitors and records any future row-level changes to this data.

Note

Real-Time hub is currently in preview.

Prerequisites

Note

Multiple tables CDC isn't supported.

Enable CDC in your Azure Database for PostgreSQL

To enable CDC in your Azure Database for PostgreSQL, follow these steps based on your deployment type.

Azure Database for PostgreSQL single server

  1. Go to the Replication page on the Azure portal.

  2. Change the replication rule to Logical.

    A screenshot of enabling CDC for a single server deployment.

Azure Database for PostgreSQL flexible server

  1. On your Azure Database for PostgreSQL flexible server page in the Azure portal, select Server parameters in the navigation menu.

  2. On the Server parameters page:

    • Set wal_level to logical.

    • Update the max_worker_processes to at least 16.

      A screenshot of enabling CDC for a flexible server deployment.

  3. Save the changes and restart the server.

  4. Confirm that your Azure Database for PostgreSQL flexible server instance allows public network traffic.

  5. Grant the admin user replication permissions by running the following SQL statement.

    ALTER ROLE <admin user> WITH REPLICATION;

Launch Add source experience

  1. Sign in to Microsoft Fabric.

  2. Select Real-Time hub on the left navigation bar.

  3. On the Real-Time hub page, select + Add source in the top-right corner of the page.

    Screenshot that shows how to launch Real-Time hub In Microsoft Fabric.

Use instructions from the Add PostgreSQL Database CDC as a source section.

Microsoft sources page

  1. In Real-Time hub, select Microsoft sources.

  2. In the Source drop-down list, select PostgreSQL (CDC).

  3. For Subscription, select an Azure subscription that has the resource group with your Azure MySQL Database account.

  4. For Resource group, select a resource group that has the database.

  5. For Region, select a location where your database is located.

  6. Now, move the mouse over the name of the PostgreSQL Database CDC source that you want to connect to Real-Time hub in the list of databases, and select the Connect button, or select ... (ellipsis), and then select the Connect button.

    Screenshot that shows the Microsoft sources page with filters to show Azure MySQL CDC and the connect button.

    To configure connection information, use steps from the Add Azure MySQL DB CDC as a source section. Skip the first step of selecting Azure MySQL DB CDC as a source type in the Add source wizard.

Add Azure Database for PostgreSQL CDC as a source

  1. On the Select a data source screen, select Azure DB for PostgreSQL (CDC).

    A screenshot of selecting PostgreSQL DB (CDC).

  2. On the Connect page, select Go to resource to navigate to the Azure PostgreSQL database. Take a note of the server name on the Overview page. It's in the following format: myazurepostgresqlserver.postgres.database.azure.com.

    Screenshot that shows the Connect page with Go to resource link highlighted.

  3. On the Connect page, select New connection.

    Screenshot that shows the Connect page for an Azure PostgreSQL database with New connection link highlighted.

  4. In the Connection settings section, enter the following information.

    • Server: The server address of your PostgreSQL database, for example my-pgsql-server.postgres.database.azure.com.

    • Database: The database name, for example my_database.

      Screenshot that shows the Connection settings section for the Azure PostgreSQL database connector.

  5. Scroll down, and in the Connection credentials section, follow these steps.

    1. For Connection name, enter a name for the connection.

    2. For Authentication kind, select Basic.

      Note

      Currently, Fabric event streams supports only Basic authentication.

    3. Enter Username and Password for the database.

    4. Select Connect.

      Screenshot that shows the Connection credentials section for the Azure PostgreSQL database connector.

  6. Now, on the Connect page, do these steps:

    1. For table, Enter a table name.

    2. For Port, enter the port number or leave the default value: 5432.

    3. For eventstream name, enter a name for the eventstream. The wizard creates an eventstream with the selected PostgreSQL Database CDC as a source.

    4. The Stream name is automatically generated for you by appending -stream to the name of the eventstream. You see this stream on the All data streams page when the wizard finishes.

    5. Select Next at the bottom of the page.

      Screenshot that shows the Connect page filled for the Azure PostgreSQL database connector.

  7. On the Review and create screen, review the summary, and then select Create source.

    Screenshot that shows the Review and create page filled for the Azure PostgreSQL database connector.

View data stream details

  1. On the Review and create page, if you select Open eventstream, the wizard opens the eventstream that it created for you with the selected PostgreSQL Database CDC as a source. To close the wizard, select Close at the bottom of the page.

    Screenshot that shows the Review and create success page.

  2. In Real-Time hub, select All data streams. To see the new data stream, refresh the All data streams page.

    Screenshot that shows the Real-Time hub All data streams page with the stream based on PostgreSQL Database CDC source.

Related content

To learn about consuming data streams, see the following articles: