Edit

Share via

Build a real-time dashboard in Microsoft Fabric using MQTT data from the MQTT broker

  • Article

Important

Azure IoT Operations Preview – enabled by Azure Arc is currently in preview. You shouldn't use this preview software in production environments.

You'll need to deploy a new Azure IoT Operations installation when a generally available release is made available. You won't be able to upgrade a preview installation.

See the Supplemental Terms of Use for Microsoft Azure Previews for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

In this walkthrough, you build a real-time Power BI dashboard in Microsoft Fabric using simulated MQTT data that's published to the MQTT broker. The architecture uses the MQTT broker's Kafka connector to deliver messages to an Event Hubs namespace. Messages are then streamed to a Kusto database in Microsoft Fabric using an eventstream and visualized in a Power BI dashboard.

Azure IoT Operation Preview - enabled by Azure Arc can be deployed with the Azure CLI, Azure portal or with infrastructure-as-code (IaC) tools. This tutorial uses the IaC method using the Bicep language.

Prepare your Kubernetes cluster

This walkthrough uses a virtual Kubernetes environment hosted in a GitHub Codespace to help you get going quickly. If you want to use a different environment, all the artifacts are available in the explore-iot-operations GitHub repository so you can easily follow along.

  1. Create the Codespace, optionally entering your Azure details to store them as environment variables for the terminal.

    Open in GitHub Codespaces

  2. Once the Codespace is ready, select the menu button at the top left, then select Open in VS Code Desktop.

    Screenshot of open VS Code on desktop.

  3. Connect the cluster to Azure Arc.

Deploy edge and cloud Azure resources

  1. Create a Microsoft Fabric Workspace.

  2. Create a Microsoft Fabric Lakehouse.

  3. A single Bicep template file from the explore-iot-operations repository deploys all the required dataflows and dataflow endpoints resources Bicep File to create Dataflow. Download the template file and replace the values for customLocationName, aioInstanceName, schemaRegistryName, opcuaSchemaName, eventGridHostName, and persistentVCName.

  4. Deploy the resources using the az stack group command in your terminal:

az stack group create --name MyDeploymentStack --resource-group $RESOURCE_GROUP --template-file /workspaces/explore-iot-operations/<filename>.bicep --action-on-unmanage 'deleteResources' --deny-settings-mode 'none' --yes

Important

The deployment configuration is for demonstration or development purposes only. It's not suitable for production environments.

Send test MQTT data and confirm cloud delivery

  1. Simulate test data by deploying a Kubernetes workload. The pod simulates a sensor by sending sample temperature, vibration, and pressure readings periodically to the MQTT broker using an MQTT client. Execute the following command in the Codespace terminal:

    kubectl apply -f tutorials/mq-realtime-fabric-dashboard/simulate-data.yaml

  2. The Kafka north-bound connector is preconfigured in the deployment to pick up messages from the MQTT topic where messages are being published to Event Hubs in the cloud.

  3. After about a minute, confirm the message delivery in Event Hubs metrics.

    Screenshot of confirming Event Hubs messages.

Create and configure Microsoft Fabric event streams

  1. Create a KQL Database.

  2. Create an eventstream in Microsoft Fabric.

    1. Add the Event Hubs namespace created in the previous section as a source.

    2. Add the KQL Database created in the previous step as a destination.

  3. In the wizard's Inspect step, add a New table called sensor_readings, enter a Data connection name and select Next.

  4. In the Preview data tab, select the JSON format and select Finish.

In a few seconds, you should see the data being ingested into KQL Database.

Screenshot showing eventstream ingesting success.

Create Power BI report

  1. From the KQL Database, right-click on the sensor-readings table and select Build Power BI report.

    Screenshot showing menu selection of Build Power BI report.

  2. Drag the ∑ Temperature onto the canvas and change the visualization to a line graph. Drag the EventEnqueuedUtcTime column onto the visual and save the report.

    Screenshot showing save dialog for a Power BI report.

  3. Open the Power BI report to see the real-time dashboard, you can refresh the dashboard with latest sensor reading using the button on the top right.

    Screenshot of a Power BI report.

In this walkthrough, you learned how to build a real-time dashboard in Microsoft Fabric using simulated MQTT data that is published to the MQTT broker.