Azure IoT hub Message Routing with blob index tags

Question

Hello Community,

I have a setup which consists of devices sending data to Azure cloud IoT Hub using message routing (to storage endpoints) which land up as blobs in a container. The frequency of data push is high. On the other end, I want to be able to query my blob container to pull files based on specific dates.

I came across blob index tags which look like a promising solution to query and is supported by the Azure SDK for .net.

I was thinking to add tags to each blob ex: processedDate: <dd/mm/yyyy>, which would help me query on the same later.

I found out that while uploading the blobs manually it is possible to add the tags but not sure how to go about or where to configure the same in the message routing flow where blobs are created on the fly. So I am looking for a solution to add those tags in flight as they are being pushed on to the container.

Any help on this will be much appreciated.

Thanks much!

Answer 1

Hello @Karla Kassen ,

the Azure IoT Hub supports routing endpoints of type Storage Account.

This basically fills separate blob files based on time and size limits.

You are not in control of the blob index tags this way.

If you want to use this, you must write your own logic.

This could be an Azure Function capable of ingesting IoT Hub messages (using the IoT Hub trigger), temporary storage (Functions are stateless by default), and saving the messages in blobs with the additional tags.

Because you are talking about many messages, you should consider running your own application service plan instead of a consumption plan.

Have you considered other ways of storage? Have you looked into Azure Data Explorer?

This is a feasible alternative for the cold and warm path if you can overcome the startup costs.

----------

If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.