How to implement realtime Communication between Device and Cloud over IoT Hub

Question

Hello,

i would like to implement a low-delay (near-realtime) control loop over Azure IoT Hub.

Control Loop:
The microcontroller sends the current position of an actuator to the IoT Hub.

Input:
{"deviceId":"MyDevice","position_x":14.00,"position_y":24.00,"position_z":75.00}

The cloud service automatically execute a short calculation and send a response to the microcontroller with the new velocity.

Return:
{"deviceId":"MyDevice","velocity":3.00}

The service should run through this loop with a minimum of delay.

Current status:

As a first Step i created an IoT Hub and ran the given Quickstart-Examples (in python).
(https://learn.microsoft.com/en-us/azure/iot-hub/quickstart-send-telemetry-python)

In the next stage I used the MSP432E401Y Microcontroller from Texas Instruments instead of the SimulatedDevice.py.

Current Status:

Next steps and Question:
Is it possible to revieve the data and return it immediately?

How is it possible to "push" this file/script to iot hub? Or is this required another service and linking to another endpoint? If I need another service, which one is suitable for the descriped application?

Goal:

I hope somebody can help me.

With best regards
Patrick

Answer 1

Hello @Patrick Schneider ,

your architecture is very challenging :-) My first reaction is: what is 'real-time'?

You are talking about sending data to the cloud, ingesting it, and turn it into an action command (after executing some logic) and send that back to the device.

The IoT Hub counts every message sent as part of the billing strategy.

Still, there is no guarantee the message will actually arrive in the cloud, the network could be down, certificates expire, somebody forgets to pay some bills or somebody runs over the router (or the IoT Hub ran out of message for that day).

Running that logic locally on your device would be a far better idea!

This leaves the question: how do I get the logic on the device?

Azure IoT has a number of ways of sending C2D messages to a device, from the cloud.

There are desired/reported properties, there are direct methods. or even automatic device configuration at scale.

All of these ways of communication can be used for sending commands to devices but real-time communication is hard to justify and it also comes with a price.

So please investigate executing logic locally which can be updated from the cloud.

Answer 2

Hello @PatrickScheider-6088,

good to hear the (dis-)advantages are clear.

So, you need 'something' between the IoT Hub which ingests the telemetry and the Azure Function which makes a decision.

Basically, this can be done using EventGrid communication or the IoT Hub internal routing.

The first one is just following the EventGrid pattern.

The IoT Hub routing falls apart in two steps:

1. there is a built-in eventhub compatible endpoint. This can be used as input/trigger by eg. Azure Stream Analytics or Azure Functions.
2. incoming messages can be routed to Azure resources like event hub or message bus using IoT Hub routing. This is great in combination with message enrichment and routing rules. Eg. the event hub messages can be picked up by your Azure function.

So it depends on the extra routing rules and message enrichment.

Be aware that using the routing rules/endpoints disable the built-in endpoint. You have to add an extra route for the built-in endpoint (which then can make use of eg. message enrichment).

Also, check the consumer groups on the IoT Hub. If multiple consumers use the same consumer group (like the default available $default ) this will give a collision. Please give each consumer its own consumer group.

I'm not aware of what you mean by the Germany West Central region not supporting events in IoT Hub. I myself use the West Europe region with all features available (it is located near Amsterdam, The Netherlands.)

Answer 3

Hello @Sander van de Velde ,

I am aware that a control loop over the cloud has several disadvantages and my work is intended to show whether and to what extent a time-relevant regulation of a process via the cloud with the current state of the art is possible.

In a future step, this control loop is to be divided into a highly available and time-relevant part (Edge), as well as a latency tolerant rule circle (Cloud). But that`s another task.

As I understand it i have to add a routing to for example an Azure Function:

Possible links between the IoT Hub and the Azure function are:

• Event Hub (Message Routing)
  or
• Event Grid (Event)

Is this possible? And if:

1) The IoT Hub has an integrated built-in endpoint (for event hub). Is it possible to use the built-in endpoint as Azure Event hub trigger for the azure function?
2) My Region (Germany West Central) does not yet support events in IoT Hub. What region supports events in IoT Hub and is near my current selection?

Answer 4

Hello @Sander van de Velde ,

thank you very much for the quick and detailed answer, this helped!

I would like to try both methods:

1) Use Eventgrid
I implemented the EventGrid pattern like this:

function.json:
{
"scriptFile": "init.py",
"bindings": [
{
"type": "eventGridTrigger",
"name": "eventGridEvent",
"direction": "in"
}
]
}

init.py:
import json
import logging
import azure.functions as func
import datetime

import random  
import sys  
from azure.iot.hub import IoTHubRegistryManager  


MESSAGE_COUNT = 1  
AVG_WIND_SPEED = 10.0  
MSG_TXT = "{\"service client sent a message\": %.2f}"  

CONNECTION_STRING = "HostName=XXXX.azure-devices.net;SharedAccessKeyName=service;SharedAccessKey=XXXXX"  
DEVICE_ID = "XXXX"  

def iothub_messaging_sample_run():  
    try:  
        # Create IoTHubRegistryManager  
        registry_manager = IoTHubRegistryManager(CONNECTION_STRING)  

        for i in range(0, MESSAGE_COUNT):  
            print ( 'Sending message: {0}'.format(i) )  
            data = MSG_TXT % (AVG_WIND_SPEED + (random.random() * 4 + 2))  

            props={}  
            # optional: assign system properties  
            props.update(messageId = "Return_message_%d" % i)  
            props.update(correlationId = "correlation_%d" % i)  
            props.update(contentType = "application/json")  

            # optional: assign application properties  
            prop_text = "PropMsg_%d" % i  
            props.update(testProperty = prop_text)  

            registry_manager.send_c2d_message(DEVICE_ID, data, properties=props)  

    except Exception as ex:  
        print ( "Unexpected error {0}" % ex )  
        return  
    except KeyboardInterrupt:  
        print ( "IoT Hub C2D Messaging service sample stopped" )  

def main(eventGridEvent: func.EventGridEvent) -> None:  

    result = json.dumps({  
        'id': eventGridEvent.id,  
        'data': eventGridEvent.get_json(),  
        'topic': eventGridEvent.topic,  
        'subject': eventGridEvent.subject,  
        'event_type': eventGridEvent.event_type,  
    })  

    logging.info('Python EventGrid trigger processed an event: %s', result)  

    iothub_messaging_sample_run()  

    msg_data = eventGridEvent.get_json()  
    logging.info(msg_data)  

    if 'properties' in msg_data and 'position' in msg_data['properties']:  
        logging.info(msg_data['properties']['position'])  

As a first step i printed the position and returned a simulated value. It works. But i am not sure this is good and efficient. Is it better or possible to return it to the IoT Hub and so on to the device via an EventGrid output binding?

Likes this:

function.json

{  
  "scriptFile": "__init__.py",  
  "bindings": [  
    {  
      "type": "eventGridTrigger",  
      "name": "eventGridEvent",  
      "direction": "in"  
    },  
    {  
      "type": "eventGrid",  
      "name": "outputEvent",  
      "topicEndpointUri": "MyEventGridTopicUriSetting",  
      "topicKeySetting": "MyEventGridTopicKeySetting",  
      "direction": "out"  
    }  
  ],  
  "disabled": false  
}  

init.py:
import json
import logging
import azure.functions as func
import datetime

def main(eventGridEvent: func.EventGridEvent,
outputEvent: func.Out[func.EventGridOutputEvent]) -> None:

result = json.dumps({  
    'id': eventGridEvent.id,  
    'data': eventGridEvent.get_json(),  
    'topic': eventGridEvent.topic,  
    'subject': eventGridEvent.subject,  
    'event_type': eventGridEvent.event_type,  
})  

logging.info('Python EventGrid trigger processed an event: %s', result)  

# ToDo: Write correct outputEvent.set(...)  

outputEvent.set(  
    func.EventGridOutputEvent(  
        id="test-id",  
        data={"tag1": "value1", "tag2": "value2"},  
        subject="test-subject",  
        event_type="test-event-1",  
        event_time=datetime.datetime.utcnow(),  
        data_version="1.0"))  


If this is possible. Where is the "topicEndpointUri" and the "topicKeySetting" from the function.json file to find?   

2) Using build-in eventhub compatible endpoint

Could you give me an introduction bechause i tried it but i am very confused how to do it.

Thank you very much.

P.S. With the region i meant that the option to add an event subscription to the IoT hub wasn't availible. But from this day this option is unlocked. Not all regions offer all features.

With best regards
Patrick