Add MySQL Database CDC as source in Real-Time hub (preview)

This article describes how to add MySQL Database Change Data Capture (CDC) as an event source in Fabric Real-Time hub.

The Azure MySQL Database Change Data Capture (CDC) connector allows you to capture a snapshot of the current data in an Azure MySQL database. You specify the tables to be monitored and get alerted when any subsequent row-level changes to the tables. Once the changes are captured in a stream, you can process this CDC data in real-time and send it to different destinations within Fabric for further processing or analysis.

Note

Real-Time hub is currently in preview.

Prerequisites

• Get access to the Fabric premium workspace with Contributor or above permissions.
• Created an instance of Azure Database for MySQL – Flexible Server.

Set up MySQL database

The Azure MySQL database connector uses the Debezium MySQL connector to capture changes in your MySQL Database. You must define a MySQL user with permissions on all databases that connector monitors. For step-by-step instructions, see Set up MySQL DB.

Enable the binlog

You must enable binary logging for MySQL replication. The binary logs record transaction updates for replication tools to propagate changes. For example, Azure Database for MySQL.

1. In the Azure portal, navigate to your Azure MySQL database.

2. On the left navigation menu, select Server parameters.

3. Configure your MySQL server with the following properties.

   • binlog_row_image: Set the value to full.
   • binlog_expire_logs_seconds: The number of seconds for automatic binlog file removal. Set the value to match the needs of your environment. For example, 86400.

   

Get events from Azure MySQL Database (CDC)

You can get events from an Azure MySQL Database CDC into Real-Time hub in one of the ways:

• Using the Get events experience
• Using the Microsoft sources tab

Launch Get events experience

1. Switch to the Real-Time Intelligence experience in Microsoft Fabric. Select Microsoft Fabric on the left navigation bar, and select Real-Time Intelligence.

   

2. Select Real-Time hub on the left navigation bar.

   

3. On the Real-Time hub page, select + Get events in the top-right corner of the page.

   

Use instructions from the Add Azure MySQL Database CDC as a source section.

Microsoft sources tab

1. In Real-Time hub, switch to the Microsoft sources tab.

2. In the Source drop-down list, select Azure MySQL DB (CDC).

3. For Subscription, select an Azure subscription that has the resource group with your Azure MySQL Database account.

4. For Resource group, select a resource group that has the database.

5. For Region, select a location where your database is located.

6. Now, move the mouse over the name of the Azure MySQL DB CDC source that you want to connect to Real-Time hub in the list of databases, and select the Connect button, or select ... (ellipsis), and then select the Connect button.

   

   To configure connection information, use steps from the Add Azure MySQL DB CDC as a source section. Skip the first step of selecting Azure MySQL DB CDC as a source type in the Get events wizard.

Add Azure MySQL Database CDC as a source

1. On the Select a data source screen, select MySQL DB (CDC).

   

2. On the Connect page, select Go to resource to navigate to the Azure PostgreSQL database. Take a note of the server name on the Overview page. It's in the following format: mysqlserver.mysql.database.azure.com.

   

3. On the Connect page, select New connection.

   

4. In the Connection settings section, do these steps:

   1. For Server, enter the URI for your Azure MySQL server.

   2. For Database, enter the name of your database.

      

5. In the Connection credentials section, do these steps:

   1. For Connection, select if there's an existing connection to the MySQL database. If not, keep the default value: Create new connection.

   2. For Authentication kind, select Basic. Currently, only Basic authentication is supported.

   3. Enter values for User name and Password.

   4. Specify whether you want to use an encrypted connection.

   5. Select Connect.

      

6. Back on the Connect page, do these steps:

   1. Enter the table name.
   2. Enter the server ID.
   3. Enter the port number or keep the default value.

7. In the Stream details section to the right, do these steps:

   1. Select Fabric workspace where you want to save this connection and the eventstream that the wizard creates.

   2. Enter a name for the eventstream.

   3. The name of the stream in Real-Time hub is automatically created for you.

      

8. Now, select Next at the bottom of the page.

9. On the Review and create, review settings, and select Create source.

   

View data stream details

1. On the Review and create page, if you select Open eventstream, the wizard opens the eventstream that it created for you with the selected Azure MySQL DB CDC as a source. To close the wizard, select Close or X* in the top-right corner of the page.

   

2. In Real-Time hub, switch to the Data streams tab of Real-Time hub. Refresh the page. You should see the data stream created for you. For detailed steps, see View details of data streams in Fabric Real-Time hub.

Related content

To learn about consuming data streams, see the following articles:

• Process data streams
• Analyze data streams
• Set alerts on data streams