How to ingest dynamic data to ADX?

Question

I am connecting to Azure IoT Hub through Azure Data Explorer (ADX) to get JSON-formatted streaming data. However, this JSON is in a nested format, and the fields of the nested objects are not fixed. How can I flatten and store it in ADX?

The JSON data like follows:

{

"type":"fruit",

"recordTime":"2023-11-09 11:09:21",

"apple":{

"weight":200,

"price":3.15

}

And the apple may have more property.I want store it in a table on ADX like follows: apple: type,recordTime,weight,price,...

Answer 1

Hello @jane,

welcome to this moderated Azure community forum.

When ingesting data from an IoT Hub, you need to create a mapping for the data connection. Check out this blog post for an example.

Is this message mapped to a dynamic column names like 'data'?

In that case, you can access the weight:

tableName
| project weight = data.apple.weight

If you want to map it directly to a typed column, you need to check the mapping like:

'{"Column": "weight", "Properties": {"Path": "$.apple. Weight"}},'

Please check out that article for a full story.

---

If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.