Edit

Add an MQTT topic as source in Real-Time hub (Preview)

This article describes how to add an MQTT topic as an event source in Fabric Real-Time hub.

Message Queuing Telemetry Transport (MQTT) is a lightweight publish-subscribe messaging protocol designed for constrained environments and widely used for Internet of Things (IoT) scenarios. The MQTT Connector for Fabric Eventstream allows you to connect to any MQTT broker and ingest messages directly into Eventstream, enabling real-time transformation and routing to various destinations within Fabric for analytics and insights.

Prerequisites

Before you start, you must complete the following prerequisites:

  • A workspace with Fabric capacity or Fabric Trial workspace type.

  • Access to the workspace with Contributor or higher workspace roles.

  • Ensure that the MQTT broker is publicly accessible and not restricted by a firewall or a virtual network. If it resides in a protected network, connect to it by using Eventstream connector virtual network injection.

  • If you plan to use TLS/mTLS settings, make sure the required certificates are available in an Azure Key Vault:

    • Import the required certificates into Azure Key Vault in .pem format.
    • The user who configures the source and previews data must have permission to access the certificates in the Key Vault (for example, Key Vault Certificate User or Key Vault Administrator).
    • If the current user doesn't have the required permissions, data can't be previewed from this source in Eventstream.

Depending on your scenario, you can also use one of the following approaches to stream MQTT data to Eventstream:

Data sources page

  1. Sign in to Microsoft Fabric.

  2. If you see Power BI at the bottom-left of the page, switch to the Fabric workload by selecting Power BI and then by selecting Fabric.

    Screenshot that shows how to switch to the Fabric workload.

  3. Select Real-Time on the left navigation bar.

    Screenshot that shows how to launch Connect to data source experience.

  4. The Streaming data page opens by default. Click on the Add data button to get to the Add data page.

    Screenshot that shows the Add data page in the Real-Time hub.

    You can also get to the Add data page directly by selecting the Add data option in the left navigation bar.

    Screenshot that shows the Connect data source button.

Select MQTT as data source type

On the Data sources page, search for MQTT, and select Connect on the MQTT tile.

Screenshot that shows the Data sources page with Connect on the MQTT tile selected.

Add MQTT topic as a source

  1. On the Connect page, select New connection.

    Screenshot that shows the Connect page the New connection link highlighted.

    If there's an existing connection to your MQTT source, select that existing connection.

  2. To create a new cloud connection, enter the following information:

    Screenshot that shows the Connection settings section.

    1. MQTT Broker URL: enter the URL of your MQTT broker. The supported protocols are ssl://, wss://, and tcp://.

      Note

      The MQTT source supports Transport Layer Security (TLS) or Secure Sockets Layer (SSL) secured MQTT connections between your MQTT broker and Eventstream. TLS or SSL connections are only supported if the server certificate is signed by a Certificate Authority (CA) included in the trusted CA list.

    2. Connection name: enter a name for the connection to the MQTT.

    3. Enter the Username and Password of your MQTT broker.

    4. Select Connect.

  3. For Topic name, enter the MQTT topic to subscribe to. The connector supports only a single topic.

  4. For Version, select either V5 or V3 based on your MQTT broker's protocol version.

  5. If your MQTT broker requires mTLS, expand TLS/mTLS settings and configure the following options as needed.

    • Trust CA certificate: Enable this option to configure the server CA certificate. Select your subscription, resource group, and key vault, and then provide the certificate name.
    • Client certificate and key: Enable this option to configure the client certificate and key.
      • Use the same CA certificate key vault: Select this checkbox when both certificates are stored in the same key vault. Then provide the certificate name.
      • If you don't select this checkbox, select the subscription, resource group, and key vault, and then provide the certificate name.

    Note

    TLS/mTLS settings in this section are currently in preview.

    For sources in a private network, ensure that the Azure Key Vault containing your certificates is connected to the Azure virtual network used by the streaming virtual network data gateway for Eventstream connector virtual network injection (for example, via a private endpoint).

    Screenshot that shows the Configure connection settings page.

TLS/mTLS certificate requirements

If you configured TLS/mTLS settings, refer to this section for certificate format specifications and common configuration mistakes when uploading to Azure Key Vault.

Certificate chain

Certificate Key size Signed by Purpose
CA certificate 4096-bit RSA Self-signed Trust anchor - the broker verifies client certificates against this CA.
Server certificate 2048-bit RSA CA Broker identity - the client verifies the broker is who it claims to be.
Client certificate 2048-bit RSA CA Client identity - the broker verifies that the connector is authorized.

Server certificate SAN requirements

The server certificate must include the broker's IP address and DNS name in the Subject Alternative Name (SAN) to pass hostname verification (ssl.endpoint.identification.algorithm=https):

subjectAltName:
  DNS.1 = {broker FQDN}
  DNS.2 = localhost
  IP.1  = {broker public IP}
  IP.2  = 127.0.0.1

Upload certificates to Azure Key Vault

Certificates are uploaded as Azure Key Vault certificate objects in PEM format. The PEM bundle file is certificate + private key concatenated in one file:

-----BEGIN CERTIFICATE-----
MIIExjCCA...
-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIB...
-----END RSA PRIVATE KEY-----

Use an import policy that matches the key properties:

{
  "secretProperties": {
    "contentType": "application/x-pem-file"
  },
  "keyProperties": {
    "exportable": true,
    "keyType": "RSA",
    "keySize": 4096,
    "reuseKey": false
  },
  "issuerParameters": {
    "name": "Unknown"
  }
}

To import the certificate, run the following command:

az keyvault certificate import \
  --vault-name {kvName} \
  --name {certName} \
  --file {pemBundleFile} \
  --policy @{policyFile}

Common mistakes

Avoid Do this instead
Upload as PKCS#12/PFX Use PEM format with contentType: application/x-pem-file.
Upload certificate without private key The PEM bundle must contain both the certificate and the key.
Set keySize: 2048 for a 4096-bit key The keySize value must match the actual key size.
Set issuerParameters.name: "Self" Use "Unknown" for externally signed certificates.
Use Windows line endings (CRLF) The PEM file must use Unix line endings (LF only).

Stream or source details

  1. On the Connect page, follow one of these steps based on whether you're using Eventstream or Real-Time hub.

    • Eventstream:

      In the Source details pane to the right, follow these steps:

      1. For Source name, select the Pencil button to change the name.

      2. Notice that Eventstream name and Stream name are read-only.

    • Real-Time hub:

      In the Stream details section to the right, follow these steps:

      1. Select the Fabric workspace where you want to create the eventstream.

      2. For Eventstream name, select the Pencil button, and enter a name for the eventstream.

      3. The Stream name value is automatically generated for you by appending -stream to the name of the eventstream. This stream appears on the real-time hub's All data streams page when the wizard finishes.

  2. Select Next at the bottom of the Configure page.

Review and connect

On the Review + connect screen, review the summary, and select Add (Eventstream) or Connect (Real-Time hub).

View data stream details

  1. On the Review + connect page, if you select Open eventstream, the wizard opens the eventstream that it created for you with the selected MQTT source. To close the wizard, select Close at the bottom of the page.
  2. You should see the stream in the Recent streaming data section of the Real-Time hub home page. For detailed steps, see View details of data streams in Fabric Real-Time hub.

Related content

To learn about consuming data streams, see the following articles:

  • Last updated on