Some data points seem to disappear downstream when using Event hubs

Question

Some data points seem to disappear downstream when using Event hubs

Kulkarni, Yash 0

Hi!

Let me explain my current setup to you -

Setup diagnostic rule in AKS to send log data to a log analytics workspace (LAW)
Setup a data export rule from to an event hub
Data goes from event hub to ASA (azure stream analytics)
And from ASA to blob storage

I am trying to check the 'Perf' table which has CPU & memory usage & limit details for my deployed AKS services. However, I run into the following issue -

I can see that the CPU & memory limits are present in the LAW (when checking via Azure monitor), but these rows aren't visible downstream i.e. in the blob storage. I have checked every stage of my soln to check if there are any data filters, and there are none. In LAW, I can see the following data -

User's image

But in my blob storage data (or even in the raw event hub data), the only counter names I see are memoryRssByter, cpuUsageNanoCores & restartTimeEpoch.

Can someone pls help!!

FYI - Yes, I checked the data export rule in LAW, I am just sending the Perf table as is without modifications.

I have only 1 partition in the event hub.

Also, I checked to see the frequency of these "limit" counters, and seems like there are alot of rows (~1000) every 5mins, but I have no rows in the blob yet.

J N S S Kasyap 3,780 Reputation points Microsoft External Staff Moderator

2025-06-03T08:25:02.3466667+00:00

Just checking in to see if the below answer provided by @Jose Benjamin Solis Nolasco helped.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

1 answer

Your answer

J N S S Kasyap 3,780 Reputation points Microsoft External Staff Moderator

2025-06-03T08:25:02.3466667+00:00

Just checking in to see if the below answer provided by @Jose Benjamin Solis Nolasco helped.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 1

Hello Kulkarni, I hope you are doing well.

Thanks for reaching out to Microsoft Q&A

I have seen this case before you can take a look at someone asking a similar question

https://learn.microsoft.com/en-us/answers/questions/2147758/debugging-data-loss-in-stream-analytics-to-event-h

Please check this guide to troubleshoot the issue;

Use AzureDiagnostics instead of Perf directly if possible Some of the limit counters (like memoryLimitBytes, cpuLimitNanoCores) show up in AzureDiagnostics logs more reliably.
Enable capture on Event Hub for testing Temporarily enable capture to Azure Blob directly from Event Hub, inspect raw events for limits data.
Explicitly define ASA schema In ASA, manually define input schema and include the missing counters like memoryLimitBytes, cpuLimitNanoCores.
Log all incoming events in ASA Add an additional output for raw input logging in ASA to a separate Blob or Table to help isolate schema/filter issues.
Temporarily increase Event Hub partition count Even if only for testing, try scaling to 2-4 partitions, especially if ingestion is near the throughput unit limit.
Recheck diagnostic data export timing Limits might be present only every few intervals, so your export pipeline must stay in sync with the full diagnostic interval range.
Consider Azure Monitor Metrics exporter For precision, sometimes using Azure Monitor Metrics → Event Hub (instead of diagnostic settings) captures limits more reliably.

😊 If my answer helped you resolve your issue, please consider marking it as the correct answer. This helps others in the community find solutions more easily. Thanks!

Jose Benjamin Solis Nolasco 3,511 Reputation points

2025-06-04T14:24:18.7+00:00

Kulkarni, Yash Just following up, Do you need more guidance or assistance ?

Share via

Some data points seem to disappear downstream when using Event hubs

1 answer

Your answer