Tutorial: Configure MQTT bridge between Azure IoT MQ Preview and Azure Event Grid

Important

Azure IoT Operations Preview – enabled by Azure Arc is currently in PREVIEW. You shouldn't use this preview software in production environments.

See the Supplemental Terms of Use for Microsoft Azure Previews for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

In this tutorial, you learn how to configure IoT MQ for bi-directional MQTT bridge with Azure Event Grid MQTT broker PaaS. You can use this feature to process your IoT data at the edge and in the cloud. For example, you can use IoT MQ to process telemetry data at the edge, and then bridge the data to Azure Event Grid for further processing in the cloud.

Prerequisites

Set environment variables

Sign in with Azure CLI:

az login

Set environment variables for the rest of the setup. Replace values in <> with valid values or names of your choice. A new Azure Event Grid namespace and topic space are created in your Azure subscription based on the names you provide:

# For this tutorial, the steps assume the IoT Operations cluster and the Event Grid
# are in the same subscription, resource group, and location.

# Name of the resource group of Azure Event Grid and IoT Operations cluster 
export RESOURCE_GROUP=<RESOURCE_GROUP_NAME>

# Azure region of Azure Event Grid and IoT Operations cluster
export LOCATION=<LOCATION>

# Name of the Azure Event Grid namespace
export EVENT_GRID_NAMESPACE=<EVENT_GRID_NAMESPACE>

# Name of the Arc-enabled IoT Operations cluster 
export CLUSTER_NAME=<CLUSTER_NAME>

# Subscription ID of Azure Event Grid and IoT Operations cluster
export SUBSCRIPTION_ID=<SUBSCRIPTION_ID>

Create Event Grid namespace with MQTT broker enabled

Create Event Grid namespace with Azure CLI. The location should be the same as the one you used to deploy Azure IoT Operations.

az eventgrid namespace create \
  --namespace-name $EVENT_GRID_NAMESPACE \
  --resource-group $RESOURCE_GROUP \
  --location $LOCATION \
  --topic-spaces-configuration "{state:Enabled,maximumClientSessionsPerAuthenticationName:3}"

By setting the topic-spaces-configuration, this command creates a namespace with:

  • MQTT broker enabled
  • Maximum client sessions per authentication name as 3.

The max client sessions option allows IoT MQ to spawn multiple instances and still connect. To learn more, see multi-session support.

Create a topic space

In the Event Grid namespace, create a topic space named tutorial with a topic template telemetry/#.

az eventgrid namespace topic-space create \
  --resource-group $RESOURCE_GROUP \
  --namespace-name $EVENT_GRID_NAMESPACE \
  --name tutorial \
  --topic-templates "telemetry/#"

By using the # wildcard in the topic template, you can publish to any topic under the telemetry topic space. For example, telemetry/temperature or telemetry/humidity.

Give Azure IoT MQ Preview access to the Event Grid topic space

Using az k8s-extension show, find the principal ID for the Azure IoT MQ Arc extension. The command stores the principal ID in a variable for later use.

export PRINCIPAL_ID=$(az k8s-extension show \
  --resource-group $RESOURCE_GROUP \
  --cluster-name $CLUSTER_NAME \
  --name mq \
  --cluster-type connectedClusters \
  --query identity.principalId -o tsv)
echo $PRINCIPAL_ID

Take note of the output value for identity.principalId, which is a GUID value with the following format:

d84481ae-9181-xxxx-xxxx-xxxxxxxxxxxx

Then, use Azure CLI to assign publisher and subscriber roles to IoT MQ for the topic space you created.

Assign the publisher role:

az role assignment create \
  --assignee $PRINCIPAL_ID \
  --role "EventGrid TopicSpaces Publisher" \
  --scope /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.EventGrid/namespaces/$EVENT_GRID_NAMESPACE/topicSpaces/tutorial

Assign the subscriber role:

az role assignment create \
  --assignee $PRINCIPAL_ID \
  --role "EventGrid TopicSpaces Subscriber" \
  --scope /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.EventGrid/namespaces/$EVENT_GRID_NAMESPACE/topicSpaces/tutorial

Tip

The scope matches the id of the topic space you created with az eventgrid namespace topic-space create in the previous step, and you can find it in the output of the command.

Event Grid MQTT broker hostname

Use Azure CLI to get the Event Grid MQTT broker hostname.

az eventgrid namespace show \
  --resource-group $RESOURCE_GROUP \
  --namespace-name $EVENT_GRID_NAMESPACE \
  --query topicSpacesConfiguration.hostname \
  -o tsv

Take note of the output value for topicSpacesConfiguration.hostname that is a hostname value that looks like:

example.region-1.ts.eventgrid.azure.net

Create an MQTT bridge connector and topic map resources

In a new file named bridge.yaml, specify the MQTT bridge connector and topic map configuration. Replace the example placeholder value in remoteBrokerConnection endpoint with the Event Grid MQTT hostname from the previous step. Include the port number 8883.

apiVersion: mq.iotoperations.azure.com/v1beta1
kind: MqttBridgeConnector
metadata:
  name: tutorial-bridge
  namespace: azure-iot-operations
spec:
  image: 
    repository: mcr.microsoft.com/azureiotoperations/mqttbridge
    tag: 0.1.0-preview
    pullPolicy: IfNotPresent
  protocol: v5
  bridgeInstances: 2
  logLevel: debug
  remoteBrokerConnection:
    endpoint: example.region-1.ts.eventgrid.azure.net:8883
    tls:
      tlsEnabled: true
    authentication:
      systemAssignedManagedIdentity:
        audience: https://eventgrid.azure.net
  localBrokerConnection:
    endpoint: aio-mq-dmqtt-frontend:8883
    tls:
      tlsEnabled: true
      trustedCaCertificateConfigMap: aio-ca-trust-bundle-test-only
    authentication:
      kubernetes: {}
---
apiVersion: mq.iotoperations.azure.com/v1beta1
kind: MqttBridgeTopicMap
metadata:
  name: tutorial-topic-map
  namespace: azure-iot-operations 
spec:
  mqttBridgeConnectorRef: tutorial-bridge
  routes:
    - direction: local-to-remote
      name: publish
      source: tutorial/local
      target: telemetry/iot-mq
      qos: 1
    - direction: remote-to-local
      name: subscribe
      source: telemetry/#
      target: tutorial/cloud
      qos: 1

You configure the MQTT bridge connector to:

  • Use the Event Grid MQTT broker as the remote broker
  • Use the local IoT MQ broker as the local broker
  • Use TLS for both remote and local brokers
  • Use system-assigned managed identity for authentication to the remote broker
  • Use Kubernetes service account for authentication to the local broker
  • Use the topic map to map the tutorial/local topic to the telemetry/iot-mq topic on the remote broker
  • Use the topic map to map the telemetry/# topic on the remote broker to the tutorial/cloud topic on the local broker

When you publish to the tutorial/local topic on the local IoT MQ broker, the message is bridged to the telemetry/iot-mq topic on the remote Event Grid MQTT broker. Then, the message is bridged back to the tutorial/cloud topic on the local IoT MQ broker. Similarly, when you publish to the telemetry/iot-mq topic on the remote Event Grid MQTT broker, the message is bridged to the tutorial/cloud topic on the local IoT MQ broker.

Apply the deployment file with kubectl.

kubectl apply -f bridge.yaml
mqttbridgeconnector.mq.iotoperations.azure.com/tutorial-bridge created
mqttbridgetopicmap.mq.iotoperations.azure.com/tutorial-topic-map created

Verify MQTT bridge deployment

Use kubectl to check the two bridge instances are ready and running.

kubectl get pods -n azure-iot-operations -l app=aio-mq-mqttbridge
NAME                       READY   STATUS    RESTARTS   AGE
aio-mq-tutorial-bridge-0   1/1     Running   0          45s
aio-mq-tutorial-bridge-1   1/1     Running   0          45s

You can now publish on the local broker and subscribe to the Event Grid MQTT Broker and verify messages flow as expected.

Deploy MQTT client

To verify the MQTT bridge is working, deploy an MQTT client to the same namespace as IoT MQ. In a new file named client.yaml, specify the client deployment:

apiVersion: v1
kind: Pod
metadata:
  name: mqtt-client
  namespace: azure-iot-operations
spec:
  serviceAccountName: mqtt-client
  containers:
  - image: alpine
    name: mqtt-client
    command: ["sh", "-c"]
    args: ["apk add mosquitto-clients mqttui && sleep infinity"]
    volumeMounts:
    - name: mq-sat
      mountPath: /var/run/secrets/tokens
    - name: trust-bundle
      mountPath: /var/run/certs
  volumes:
  - name: mq-sat
    projected:
      sources:
      - serviceAccountToken:
          path: mq-sat
          audience: aio-mq
          expirationSeconds: 86400
  - name: trust-bundle
    configMap:
      name: aio-ca-trust-bundle-test-only

Apply the deployment file with kubectl.

kubectl apply -f client.yaml
pod/mqtt-client created

Start a subscriber

Use kubectl exec to start a shell in the mosquitto client pod.

kubectl exec --stdin --tty mqtt-client -n azure-iot-operations -- sh

Inside the shell, start a subscriber to the IoT MQ broker on the tutorial/# topic space with mqttui.

mqttui log "tutorial/#" \
-b mqtts://aio-mq-dmqtt-frontend:8883 \
-u '$sat' \
--password $(cat /var/run/secrets/tokens/mq-sat) \
--insecure

Leave the command running and open a new terminal window.

Publish MQTT messages to the cloud via the bridge

In a new terminal window, start another shell in the mosquitto client pod.

kubectl exec --stdin --tty mqtt-client -n azure-iot-operations -- sh

Inside the shell, use mosquitto to publish five messages to the tutorial/local topic.

mosquitto_pub -h aio-mq-dmqtt-frontend -p 8883 \
-m "This message goes all the way to the cloud and back!" \
-t "tutorial/local" -u '$sat' -P $(cat /var/run/secrets/tokens/mq-sat) \
--cafile /var/run/certs/ca.crt \
--repeat 5 --repeat-delay 1 -d

View the messages in the subscriber

In the subscriber shell, you see the messages you published.

23:17:50.802 QoS:AtMostOnce  tutorial/local     Payload( 52): This message goes all the way to the cloud and back!
23:17:51.086 QoS:AtMostOnce  tutorial/cloud     Payload( 52): This message goes all the way to the cloud and back!
23:17:51.803 QoS:AtMostOnce  tutorial/local     Payload( 52): This message goes all the way to the cloud and back!
23:17:51.888 QoS:AtMostOnce  tutorial/cloud     Payload( 52): This message goes all the way to the cloud and back!
23:17:52.804 QoS:AtMostOnce  tutorial/local     Payload( 52): This message goes all the way to the cloud and back!
23:17:52.888 QoS:AtMostOnce  tutorial/cloud     Payload( 52): This message goes all the way to the cloud and back!
23:17:53.805 QoS:AtMostOnce  tutorial/local     Payload( 52): This message goes all the way to the cloud and back!
23:17:53.895 QoS:AtMostOnce  tutorial/cloud     Payload( 52): This message goes all the way to the cloud and back!
23:17:54.807 QoS:AtMostOnce  tutorial/local     Payload( 52): This message goes all the way to the cloud and back!
23:17:54.881 QoS:AtMostOnce  tutorial/cloud     Payload( 52): This message goes all the way to the cloud and back!

Here, you see the messages are published to the local IoT MQ broker to the tutorial/local topic, bridged to Event Grid MQTT broker, and then bridged back to the local IoT MQ broker again on the tutorial/cloud topic. The messages are then delivered to the subscriber. In this example, the round trip time is about 80 ms.

Check Event Grid metrics to verify message delivery

You can also check the Event Grid metrics to verify the messages are delivered to the Event Grid MQTT broker. In the Azure portal, navigate to the Event Grid namespace you created. Under Metrics > MQTT: Successful Published Messages. You should see the number of messages published and delivered increase as you publish messages to the local IoT MQ broker.

Screenshot of the metrics view in Azure portal to show successful MQTT messages.

Tip

You can check the configurations of topic maps, QoS, and message routes with the CLI extension az iot ops check --detail-level 2.

Next steps

In this tutorial, you learned how to configure IoT MQ for bi-directional MQTT bridge with Azure Event Grid MQTT broker. As next steps, explore the following scenarios:

  • To use an MQTT client to publish messages directly to the Event Grid MQTT broker, see Publish MQTT messages to Event Grid MQTT broker. Give the client a publisher permission binding to the topic space you created, and you can publish messages to any topic under the telemetry, like telemetry/temperature or telemetry/humidity. All of these messages are bridged to the tutorial/cloud topic on the local IoT MQ broker.
  • To set up routing rules for the Event Grid MQTT broker, see Configure routing rules for Event Grid MQTT broker. You can use routing rules to route messages to different topics based on the topic name, or to filter messages based on the message content.