Receiving C2D message without Azure SDK

Question

Receiving C2D message without Azure SDK

Andrea Previtali 106

Hi all,

I have a "working" architecture which allows me to get data from my devices and put them into an Azure table, I also have an API that do some stuff on this data if it is needed but this is not important at the moment.

Now I am looking for a way to send messages from Cloud to Device, honestly I don't know how to do that and all the sample I found on the net are realized with Azure SDK, my problem is this: I can't use Azure SDK on my devices since one of the system I am using is quite aged and doesn't support it.

Does anyone have a solution to my problem?

Thank you,

Andrea

AshokPeddakotla-MSFT 36,021 Reputation points Moderator

2022-09-15T05:28:44.21+00:00

@Andrea Previtali Wanted to follow up with you to understand whether the below information helped. Please let us know if there are any other queries.
If below suggestions answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.
Andrea Previtali 106 Reputation points

2022-09-16T06:04:21.31+00:00

Hi @AshokPeddakotla-MSFT !
I will elaborate on this topic between today and Monday, I had some setbacks at work and I am very busy.
I will give you my more precise feedback in the coming days.
Sorry.

Andrea

Answer accepted by question author

Sander van de Velde | MVP 37,066 MVP

Hello @Andrea Previtali ,

the Azure IoT Hub supports 'native' MQTT and acts like a broker.

It supports both D2C and C2D.

You need to know the flow and use of topics to get this working.

Check out this blog post explaining the interface with C# as reference language.

If this is not possible, you could add a 'guardian angel' a secure cloud-connected device eg. an Azure Sphere next to your original device.

Or, if you want to add more computing power next to the original device, check out Azure IoT Edge. This is a (industrial) PC running Linux/Windows and hosts modules (based on Docker container logic).

Andrea Previtali 106 Reputation points

2022-09-16T06:05:50.177+00:00

Hi @Sander van de Velde | MVP ,

I was convinced that I had responded to your comment, I apologize.
As mentioned above, I will look into this further in the coming days and give you feedback.
Thank you very much for your support.

Andrea
Andrea Previtali 106 Reputation points

2022-09-20T08:28:26.79+00:00

Thank you @Sander van de Velde | MVP , your help has been very important for me.
I'm quite new to IoT World and the article you sent me helped a lot.
Sander van de Velde | MVP 37,066 Reputation points MVP

2022-09-20T09:03:47.85+00:00

Hello @Andrea Previtali ,

nice to see we add value to your IoT journey.

Just keep asking questions here if needed. We always try to answer them.
Andrea Previtali 106 Reputation points

2022-09-20T12:45:27.817+00:00

Hello @Sander van de Velde | MVP !

Since you are always try to answer... I have another little question XD

Now I have two different C program, minimal program with no particular check.
The first one implement the pub method and it works, I'm able to run it on my PC and also on my target device (both target device, the first with Yocto and the second with Buildroot).
As I told I use the sent data to build table and stuff.

My focus now, and it was the topic of my initial question, is on a program which allow me to implement sub method.
I started from the sample I found on Microsoft Azure tutorial.
My final goal is to use sub method "read" the message that pub method posted before

To try fast the program I am using the same .pem and .key file I used for the pub program. Obviously I can't use both program in the same moment

When I send a message C2D using IoT Explorer or Azure console I am able to receive the message in my sub program and it is good.
I only have problem when the message is sent from pub program.

Shortly my situation is that:

I send C2D message using IoT Explorer: sub program works
I send C2D message using Azure console: sub program works
I try to simulate C2D message using my pub program: sub program doesn't work. I want to pub a message with a program and read this message from another program.

I don't get any error, just message doesn't arrive.
Sander van de Velde | MVP 37,066 Reputation points MVP

2022-09-20T17:16:21.53+00:00

Hello @Andrea Previtali ,

it seems you want to send a message from one device to another device.

This is where Azure IoT Hub differs from traditional MQTT brokers, devices have a secure connection to the IoT Hub and are unaware of other devices.

If you want to send a message from one device to another, you need to add background logic (like an Azure Function listening to IoT Hub messages) to route messages back to other devices.

Please check the Azure Feedback Forum if you want to submit your idea for Azure IoT.
Andrea Previtali 106 Reputation points

2022-09-21T06:16:16.747+00:00

Hello @Sander van de Velde | MVP !

Then it is as I had thought. Looking at the examples of the pub and sub functions I imagined that was the case.

They were actually tests I was doing to get a good grasp of the concepts of the pub and sub methods. For my final needs to use, a simple C2D message would suffice.
If I wanted to send the message to multiple devices how would I go about it? Do you think it is possible to write an API that would allow me to send the same message to devices that meet certain characteristics?

Thanks in advance
Sander van de Velde | MVP 37,066 Reputation points MVP

2022-09-21T14:16:36.81+00:00

Hello @Andrea Previtali ,

the background function has to decide which devices need to get a message.

Perhaps a call to a database or Azure Table Storage account is good enough for that decision.

From there, cloud messages or DeviceTwin updates are interesting for these devices.

These two C2D message solutions are async by nature. So offline devices are able to get the update later on when they connect.

Otherwise, calling a Direct Method will do a blocking call to the connected device of your choice.
Andrea Previtali 106 Reputation points

2022-09-21T15:26:48.827+00:00

Hi @Sander van de Velde | MVP

Ok, it makes sense.
I'm keep going deeper this stuff.

These two C2D message solutions are async by nature. So offline devices are able to get the update later on when they connect.

About that... What if I need to read that message more than one time?
I noticed that after sub the topic and after "reading the message", if I invoke again the sub method... I can't se the message anymore.
Is there a way to set a message as "ratained" or stuff like that?

Imagine this situation: everytime I turn on my machine, a sub method is launched and I have to read the last C2D message sent in the past.
Is possible to manage a solution for that issue?
Sander van de Velde | MVP 37,066 Reputation points MVP

2022-09-22T06:25:04.587+00:00

Hello @Andrea Previtali ,

what you wish for is beyond normal device behavior. If a message is received, it is gone from IoT Hub perspective..

Because, normally, device are expected to persist state.

But.. you could use the reported properties to put state in the cloud, you can both write and read reported properties.

Note: this comes with a cost. Device Twin changes are counted too as IoT Hub message exchange.
Andrea Previtali 106 Reputation points

2022-09-23T06:09:58.11+00:00

Hi @Sander van de Velde | MVP ,

At first really thanks for your help, with your suggestion I am building something like a POC that I will show to my team in the next days.
Starting from that we will manage a complete solution with proper checks and so on.

I'm facing up a little problem. I'm sorry if what I'm about to write may seem stupid to you but, as I told, I'm new to this world XD.
In the past days I compiled two diffent C program (my final goal is to implement alla the methods in an unique solution), one performing pub method and the second performing sub method.

The first one is used to send telemetry data from device to IoT Hub, the second one is used to receive message from the could.

the current scenario is this: every 10 seconds I launch the pub method and send data to the cloud, once a minute instead I invoke the sub method and check if any messages have been sent from the cloud to the device.
In this situation I very often miss the message going from the cloud to the device.
On the other hand, if I only invoke the sub method, without using the C program that implements the pub method, every message arrives without a problem. In this condition, as you suggested, even if I send a message when the device is turned off, once it is turned on it manages to detect the message.

My conclusion is that by opening and closing the connection somehow I compromise the status of the message I sent previously.
I don't know if I made myself clear, do you have any thoughts on this?

0 Votes0 · More
Sander van de Velde | MVP 37,066 Reputation points MVP

2022-09-23T06:28:55.16+00:00

Hi @Sander van de Velde | MVP ,

async communication (cloud messages, Desired properties) stays on the IoT Hub until picked up by the device.

So, as shown in the blog post, you need to decide when to sync.

eg. on startup, you check for new cloud messages sent (put on the queue in the IoT Hub) while the device was offline. Meanwhile, you listen for new updates live.

If you do not have state on the device, you probably want to read the current desired properties.

I recommend learning about the abilities of Azure IoT Hub messaging using the SDKs first so you have a proper understanding of the capabilities of Azure IoT. Just then, try to do the same without SDK, within the limitations of your device.
Andrea Previtali 106 Reputation points

2022-09-23T07:05:48.377+00:00
Hi @Sander van de Velde | MVP ,

If I don't perform any pub action I am always able to sync and look for cloud message, even if the device is turned off. I am able to "read" the enteire queue once the device is turned on.

I have problem to read the cloud message or the cloud queue only when device send a message to cloud.

Machine turned on, only sub method: I get all the C2D messages without any problem.

Machine turned off, only sub method invocked once the machine is turned on: I get all the C2D messages without any problem (or the queue).

Machine turned on with pub method running every 10 seconds, if I invocke sub method, it doesn't sync, it seems the messages are lost.
Sander van de Velde | MVP 37,066 Reputation points MVP

2022-09-23T11:27:07.117+00:00

Hello @Andrea Previtali ,

can you test the same scenario with an SDK supporting device (e.g. in C# or Python)?

Be sure you use MQTT (not AMQP or the WebSocket variant).

Do you see the same behavior?
Andrea Previtali 106 Reputation points

2022-09-23T12:30:01.407+00:00

I'm pretty sure I am using MQTT ;) in paticular I am working with mosquitto.

I would find a unique solution for all my devices and since I have different OS. Unfortunately not all of them support Azure SDK. For this reason I would find a solution in C.
I guess I am close to the target, just have to fix some stupid thing.

Maybe the error is in my approach to the IoT Hub: Every time I launch the PUB program I open the connection to IoT Hub, I send the message and I close the connection (the C program also terminates).
I do the same thing with the SUB program. Could this scenario introduce some problem since every ten seconds I send a connection request to the IoT Hub and aftar that I abort the connection?

1 additional answer

Your answer

AshokPeddakotla-MSFT 36,021 Reputation points Moderator

2022-09-15T05:28:44.21+00:00

@Andrea Previtali Wanted to follow up with you to understand whether the below information helped. Please let us know if there are any other queries.
If below suggestions answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.
Andrea Previtali 106 Reputation points

2022-09-16T06:04:21.31+00:00

Hi @AshokPeddakotla-MSFT !
I will elaborate on this topic between today and Monday, I had some setbacks at work and I am very busy.
I will give you my more precise feedback in the coming days.
Sorry.

Andrea
Andrea Previtali 106 Reputation points

2022-09-16T06:05:50.177+00:00

Hi @Sander van de Velde | MVP ,

I was convinced that I had responded to your comment, I apologize.
As mentioned above, I will look into this further in the coming days and give you feedback.
Thank you very much for your support.

Andrea
Andrea Previtali 106 Reputation points

2022-09-20T08:28:26.79+00:00

Thank you @Sander van de Velde | MVP , your help has been very important for me.
I'm quite new to IoT World and the article you sent me helped a lot.
Sander van de Velde | MVP 37,066 Reputation points MVP

2022-09-20T09:03:47.85+00:00

Hello @Andrea Previtali ,

nice to see we add value to your IoT journey.

Just keep asking questions here if needed. We always try to answer them.
Andrea Previtali 106 Reputation points

2022-09-20T12:45:27.817+00:00

Hello @Sander van de Velde | MVP !

Since you are always try to answer... I have another little question XD

Now I have two different C program, minimal program with no particular check.
The first one implement the pub method and it works, I'm able to run it on my PC and also on my target device (both target device, the first with Yocto and the second with Buildroot).
As I told I use the sent data to build table and stuff.

My focus now, and it was the topic of my initial question, is on a program which allow me to implement sub method.
I started from the sample I found on Microsoft Azure tutorial.
My final goal is to use sub method "read" the message that pub method posted before

To try fast the program I am using the same .pem and .key file I used for the pub program. Obviously I can't use both program in the same moment

When I send a message C2D using IoT Explorer or Azure console I am able to receive the message in my sub program and it is good.
I only have problem when the message is sent from pub program.

Shortly my situation is that:

I send C2D message using IoT Explorer: sub program works
I send C2D message using Azure console: sub program works
I try to simulate C2D message using my pub program: sub program doesn't work. I want to pub a message with a program and read this message from another program.

I don't get any error, just message doesn't arrive.
Sander van de Velde | MVP 37,066 Reputation points MVP

2022-09-20T17:16:21.53+00:00

Hello @Andrea Previtali ,

it seems you want to send a message from one device to another device.

This is where Azure IoT Hub differs from traditional MQTT brokers, devices have a secure connection to the IoT Hub and are unaware of other devices.

If you want to send a message from one device to another, you need to add background logic (like an Azure Function listening to IoT Hub messages) to route messages back to other devices.

Please check the Azure Feedback Forum if you want to submit your idea for Azure IoT.
Andrea Previtali 106 Reputation points

2022-09-21T06:16:16.747+00:00

Hello @Sander van de Velde | MVP !

Then it is as I had thought. Looking at the examples of the pub and sub functions I imagined that was the case.

They were actually tests I was doing to get a good grasp of the concepts of the pub and sub methods. For my final needs to use, a simple C2D message would suffice.
If I wanted to send the message to multiple devices how would I go about it? Do you think it is possible to write an API that would allow me to send the same message to devices that meet certain characteristics?

Thanks in advance
Sander van de Velde | MVP 37,066 Reputation points MVP

2022-09-21T14:16:36.81+00:00

Hello @Andrea Previtali ,

the background function has to decide which devices need to get a message.

Perhaps a call to a database or Azure Table Storage account is good enough for that decision.

From there, cloud messages or DeviceTwin updates are interesting for these devices.

These two C2D message solutions are async by nature. So offline devices are able to get the update later on when they connect.

Otherwise, calling a Direct Method will do a blocking call to the connected device of your choice.
Andrea Previtali 106 Reputation points

2022-09-21T15:26:48.827+00:00

Hi @Sander van de Velde | MVP

Ok, it makes sense.
I'm keep going deeper this stuff.

These two C2D message solutions are async by nature. So offline devices are able to get the update later on when they connect.

About that... What if I need to read that message more than one time?
I noticed that after sub the topic and after "reading the message", if I invoke again the sub method... I can't se the message anymore.
Is there a way to set a message as "ratained" or stuff like that?

Imagine this situation: everytime I turn on my machine, a sub method is launched and I have to read the last C2D message sent in the past.
Is possible to manage a solution for that issue?
Sander van de Velde | MVP 37,066 Reputation points MVP

2022-09-22T06:25:04.587+00:00

Hello @Andrea Previtali ,

what you wish for is beyond normal device behavior. If a message is received, it is gone from IoT Hub perspective..

Because, normally, device are expected to persist state.

But.. you could use the reported properties to put state in the cloud, you can both write and read reported properties.

Note: this comes with a cost. Device Twin changes are counted too as IoT Hub message exchange.
Andrea Previtali 106 Reputation points

2022-09-23T06:09:58.11+00:00

Hi @Sander van de Velde | MVP ,

At first really thanks for your help, with your suggestion I am building something like a POC that I will show to my team in the next days.
Starting from that we will manage a complete solution with proper checks and so on.

I'm facing up a little problem. I'm sorry if what I'm about to write may seem stupid to you but, as I told, I'm new to this world XD.
In the past days I compiled two diffent C program (my final goal is to implement alla the methods in an unique solution), one performing pub method and the second performing sub method.

The first one is used to send telemetry data from device to IoT Hub, the second one is used to receive message from the could.

the current scenario is this: every 10 seconds I launch the pub method and send data to the cloud, once a minute instead I invoke the sub method and check if any messages have been sent from the cloud to the device.
In this situation I very often miss the message going from the cloud to the device.
On the other hand, if I only invoke the sub method, without using the C program that implements the pub method, every message arrives without a problem. In this condition, as you suggested, even if I send a message when the device is turned off, once it is turned on it manages to detect the message.

My conclusion is that by opening and closing the connection somehow I compromise the status of the message I sent previously.
I don't know if I made myself clear, do you have any thoughts on this?

0 Votes0 · More
Sander van de Velde | MVP 37,066 Reputation points MVP

2022-09-23T06:28:55.16+00:00

Hi @Sander van de Velde | MVP ,

async communication (cloud messages, Desired properties) stays on the IoT Hub until picked up by the device.

So, as shown in the blog post, you need to decide when to sync.

eg. on startup, you check for new cloud messages sent (put on the queue in the IoT Hub) while the device was offline. Meanwhile, you listen for new updates live.

If you do not have state on the device, you probably want to read the current desired properties.

I recommend learning about the abilities of Azure IoT Hub messaging using the SDKs first so you have a proper understanding of the capabilities of Azure IoT. Just then, try to do the same without SDK, within the limitations of your device.
Andrea Previtali 106 Reputation points

2022-09-23T07:05:48.377+00:00

Hi @Sander van de Velde | MVP ,

If I don't perform any pub action I am always able to sync and look for cloud message, even if the device is turned off. I am able to "read" the enteire queue once the device is turned on.

I have problem to read the cloud message or the cloud queue only when device send a message to cloud.

Machine turned on, only sub method: I get all the C2D messages without any problem.

Machine turned off, only sub method invocked once the machine is turned on: I get all the C2D messages without any problem (or the queue).

Machine turned on with pub method running every 10 seconds, if I invocke sub method, it doesn't sync, it seems the messages are lost.
Sander van de Velde | MVP 37,066 Reputation points MVP

2022-09-23T11:27:07.117+00:00

Hello @Andrea Previtali ,

can you test the same scenario with an SDK supporting device (e.g. in C# or Python)?

Be sure you use MQTT (not AMQP or the WebSocket variant).

Do you see the same behavior?
Andrea Previtali 106 Reputation points

2022-09-23T12:30:01.407+00:00

I'm pretty sure I am using MQTT ;) in paticular I am working with mosquitto.

I would find a unique solution for all my devices and since I have different OS. Unfortunately not all of them support Azure SDK. For this reason I would find a solution in C.
I guess I am close to the target, just have to fix some stupid thing.

Maybe the error is in my approach to the IoT Hub: Every time I launch the PUB program I open the connection to IoT Hub, I send the message and I close the connection (the C program also terminates).
I do the same thing with the SUB program. Could this scenario introduce some problem since every ten seconds I send a connection request to the IoT Hub and aftar that I abort the connection?

Answer 1

Andrea Previtali 106

Hi @Sander van de Velde | MVP ,

After some weeks I'm back again on this "issue". Do you have any suggestion about that?
Consider my ""project"" started from here: https://github.com/Azure-Samples/IoTMQTTSample/tree/master/src/Windows
The telemetry program has no problem, I can invoke it every 10 seconds and I don't have any kind of issue.

I also don't have any problem if I run Subscribe program alone: If I send a message to the device, subscribe program, once launched, is able to retrive this message, also if the machine was turned off while the messages was sent.

The problem I'm trying to solve is the following:

I send the message to the turned off machine.
I turn on the machine and I run TELEMETRY program.
I stop the Telemetry program
I run the Subscribe program: in this situation I am not able to retrive the message.

Any idea on the possible origin of my issue?

Thanks,

Sander van de Velde | MVP 37,066 Reputation points MVP

2022-10-18T10:01:11.527+00:00

Hi @Andrea Previtali ,

Can you check the number of messages queued, as seen in the new IoT Hub device list:
Andrea Previtali 106 Reputation points

2022-10-18T12:16:57.2+00:00

Hi @Sander van de Velde | MVP ,
When I send the message to the device the counter correctly goes to 1, but once i PUB from my device (so when I run the "pub" C program) the counter goes to 0.

I am honestly not skilled in this matter, for this reason I started to develop my code from Microsoft sample (oub method realized starting fron this code: https://github.com/Azure-Samples/IoTMQTTSample/blob/master/src/Shared/MQTT_Mosquitto.cpp).

I only changed the message body and the authentication, in the example SaS token is used, I preferred to use X.509 certs.

Thanks

Share via

Receiving C2D message without Azure SDK

1 additional answer

Your answer