Share via

Azure Event Hubs and Custom Headers / Properties

Harman 20 Reputation points
2023-08-11T21:43:17.2533333+00:00

I know Azure Event Grid supports custom header values - both static and dynamic via delivery properties, but trying to understand if something similar is supported in Azure Event Hubs as well.

Here is what I am trying to achieve:

  1. Delivering / ingesting IoT data with some changes via Azure Event Hubs something similar to this - done
  2. As my data leaves Azure Event Hubs, I would like to generate an event with some custom headers containing part of the data to notify that X record has been ingested into ADX (or for that matter could be delivered to any target, but specifying for clarity!)

There are no errors to report as such, I am just not sure on how to achieve this functionality and looking forward to any suggestions if anyone has implemented a similar solution.

TIA !

Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
719 questions
0 comments
Accepted answer
  1. Sander van de Velde | MVP 36,761 Reputation points MVP Volunteer Moderator
    2023-08-12T22:18:33.6933333+00:00

    Hello @Harman,

    the short answer is that EventHub messages follow this EventData format.

    This means that next to a message body, either devices or any service between the device and the EventHub van add Application properties (a property bag, key-value pairs). This could be the IoT Hub (e.g. using IoT Hub message enrichment)

    There are also system properties but these properties are reserved for Azure resources.

    Unfortunately, the Azure Data Explorer data connections (eg. IoT Hub ingest of Event Hub data ingest) do not support application properties yet!

    So, if you want to ingest application properties next to the message body, you need to transform the incoming messages using services like Azure Functions or Azure Stream Analytics.

    An alternative is using the Azure Data Explorer SDK so you can construct completely newly structured messages.

    This could also be used in an Azure Function as an alternative for the message count, eg. in combination with a static counter variable.

    If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.

    2 people found this answer helpful.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.