Telemetry data JSON body is encrypted. IoT Message route is used to route data directly to Blob storage.

Question

Hi,

So I am running Advantech device on Rasberry Pi using ModBus which then use Node Red workflow and Azure IOT hub node to connect to Azure IOT hub directly. I am able to connect to IoT hub and telemetry data is being sent successfully to iot hub. In order to validate the data, I have created a message route for blob storage end point. However, data is showing but body of the data is encrypted. It shows as

{"EnqueuedTimeUtc":"2022-12-16T13:39:10.9060000Z","Properties":{},"SystemProperties":{"connectionDeviceId":"U3435*","connectionAuthMethod":"{\"scope\":\"device\",\"type\":\"sas\",\"issuer\":\"iothub\",\"acceptingIpFilterRule\":null}","connectionDeviceGenerationId":"63806182907166438*","enqueuedTime":"2022-12-16T22:39:10.9010000Z"},"Body":"eyJUZW1******dHVyZSI6eyJUZW1wMCI6NTAwfSwiRmfghyI6eyJGZWVkMCI6MzJKLY=*"}

I have checked the system properties in iot hub for message routing which are as follow content type contentEncoding : utf: 8 and content type : application/json..

In Node Red Azure iot hub node, I am using MQTT protocol and It looks like this

msg.payload = {
'deviceId': "U3435*",
'key': "***********",
'protocol' : "mqtt",
'data' :msg.payload
}
return msg;

I also tried sending this message from Node Red

{"deviceId":"U3435*","key":"fgfdgfrdtrtrtwewghthytututuh*=","protocol":"mqtt","data":{"Tempereture":{"Temp0":100},"Humidity":{"Hmdt0":10}}}

But still no success in decrypting the body of the JSON file in stroage.

So i need to be able to read the body of the json file. Also, when I tried to use Azure stream analytics to verify the data. First, It gave error ' An AMQP error occurred (condition='amqp:unauthorized-access'). [SessionID: *****'].

Then I have created a seperate consumer group for stream analytics in iot hub which is seperate from message route to stroage end point. But when I test the input connection it gives error 'can not read input from iot hub connection string etc' but after couple of tries. It works and Stream analytics successfully runs. FYI, I am using iothub owner policy and made sure iothub input details are correct such utf-8,JSON, correct consumer group etc. But in query it still cannot read the data from iot hub and give this error '' No data was found for preview from 'iothubinput'. Make sure the input has recently received data and the correct format of those events has been selected. [SessionID: ****b*]''

How can i fix these issue? System properties snapshot is attached.

Thank you

Answer 1

Hello @Umair Akhtar Thank you for feedback. Upon further research, I found the following documentation on MQTT for Sending device-to-cloud messages.

The article mentions to append $.ct=application%2Fjson%3Bcharset%3Dutf-8 at the end of the topic in MQTT node. Please refer the below image for more details on this

Please find the below image from my MQTT node properties settings where I appended the above setting at the end of the Topic in my MQTT out node.

Here is the flow I have created to test this

My Inject node has the following message pay load

Please find the following entries created in the blob container for the above data injection from node-red

{"EnqueuedTimeUtc":"2022-12-21T02:19:15.2570000Z","Properties":{},"SystemProperties":{"connectionDeviceId":"TestDevice1","connectionAuthMethod":"{\"scope\":\"device\",\"type\":\"sas\",\"issuer\":\"iothub\",\"acceptingIpFilterRule\":null}","connectionDeviceGenerationId":"638012746782284609","contentType":"application/json;charset=utf-8","enqueuedTime":"2022-12-21T02:19:15.2570000Z"},"Body":{"temp":24,"humidity":52}}  
{"EnqueuedTimeUtc":"2022-12-21T02:19:34.9600000Z","Properties":{},"SystemProperties":{"connectionDeviceId":"TestDevice1","connectionAuthMethod":"{\"scope\":\"device\",\"type\":\"sas\",\"issuer\":\"iothub\",\"acceptingIpFilterRule\":null}","connectionDeviceGenerationId":"638012746782284609","contentType":"application/json;charset=utf-8","enqueuedTime":"2022-12-21T02:19:34.9600000Z"},"Body":{"temp":24,"humidity":52}}  

Please give this a try and let us know if you still run into any issues.

----------

Kindly accept answer or upvote if the response is helpful so that it would benefit other community members facing the same issue. I highly appreciate your contribution to the community.

Answer 2

So I have tried this mqtt out node with setting mentioned but it still did not connect for some reason. However, I was still managed to send sample data to storage which has decrypted body message now. However, Azure stream analytics still does not read the data from iot hub input and give following error "No data was found for preview from 'iothub-input'. Make sure the input has recently received data and the correct format of those events has been selected. [SessionID: XXXXXXXXXXXXX5ae9xxx]'".

I have tested the iot hub input connection and created sepearte dedicated consumer group for stream analytics in iot hub. When I test the query It says 'There is not data from iothub-input.Please make sure input source has data and try again'. I tested the input connection several times and made sure the data is flowing to iot hub and to storage route. But for some reason, stream analytics still does not read any data from iot hub.

Is there fix for this?

Or would this have something to do with Node Red configuration still? FYI, storage is now receiving decypted data in correct data foramt.

Answer 3

Some of the errors I am receiving in the stream analytics are 'No data was found for preview from 'iothubinput'. Make sure the input has recently received data and the correct format of those events has been selected.]' and ' Connection test failed : Unable to connect to IoT hub 'hubinput' using connection string. Please make sure you have configured IoT Hub correctly.' when I run test for iot hub input. Sometime it runs successfully and sometimes It give this error. In both cases, there are not input read by Stream analytics query.

What could be possible solution for this?