Mismatch in amount of data received in logs analytics workspace and DCR metrics

Ashwin Venkatesha 60 Reputation points
2024-03-28T07:47:47.7+00:00

I have defined a data collection rule and am using logs ingestion api to send data to 2 custom tables.

I have defined diagnostic settings for the DCR such that error logs are sent to logs analytics workspace.

For about an hour, I have events ingested into custom tables using logs ingestion api.

From the query in logs analytics workspaces,

let Count1 = toscalar(table('FlowLogs_CL') | summarize count());
let Count2 = toscalar(table('CustomLog_CL') | summarize count());
print TotalCount = Count1 + Count2

Answer: 324

However, from the metrics "logs rows received per min", the number is quite different

User's image

Note, that there are 0 rows dropped during ingestion, no errors reported by DCRErrorLogs

Why is it that the query results are not matching?

Azure Monitor
Azure Monitor
An Azure service that is used to collect, analyze, and act on telemetry data from Azure and on-premises environments.
2,781 questions
Microsoft Sentinel
Microsoft Sentinel
A scalable, cloud-native solution for security information event management and security orchestration automated response. Previously known as Azure Sentinel.
969 questions
{count} votes

Accepted answer
  1. AnuragSingh-MSFT 19,676 Reputation points
    2024-04-05T05:01:28.55+00:00

    Ashwin Venkatesha, thank you for the clarification. I understand the scenario now.

    The numbers as reported for "Log Rows Received per Min (Sum)" generally will not be equal to "Actual count of rows" available in the custom table. The following section discusses it in detail:

    Log Rows Received per Min (Sum) - This metric simply sums up the counter values received (which is "Log Rows Received per Min"). Notice the "per min" aggregation happening here. The sum of this aggregation is not going to be equal to the "actual count" of logs ingested. If the counter were "count of rows received", the sum would have been equal to count of rows - which is not the case here.

    Also, notice that this value changes depending on the granularity chosen from the filter above:

    User's image

    In short, these are 2 different values and should not be compared or expected to be equal.

    Hope this helps.


0 additional answers

Sort by: Most helpful