Azure Monitor How to detect/handle missing data

Wallin, Niklas 96 Reputation points
2021-02-10T17:23:34.67+00:00

I am using Log Analytics to get Performance Data from VM's using the Monitor Agent but also custom logs ingested by my application. The application runs on multiple VM's and can report a unique application instance ID for example. But if, the application or the monitor agent for some reason cannot ingest the logs, the rows will just be missing and there is "no" way to detect that.

What is the best practice to detect or visualize this (for example if an application or VM would go down and no logs are created by that instance). The only thing I can come up with is:

  1. Create a query to list all the Application Instance ID's that ever produced something in the logs as a reference and then based on that make a join with another query to detect which ones have reported a HeartBeat for the last 5 minutes? The problem is that VM's may be decommissioned, added, etc. so that the list may not be accurate.
  2. Have a separate table or log where all the expected instance id's are listed, and then from these ID's make a "join" with the query to detect heart beats within 5 minutes. Is that even possible? To have a "static" list to join with. I guess you could create another log type to just add the expected instance id's and then join?

I would think that the second option is better but not sure how or if it is even possible. Are there other/better alternatives?

Thanks

Azure Monitor
Azure Monitor
An Azure service that is used to collect, analyze, and act on telemetry data from Azure and on-premises environments.
2,938 questions
0 comments No comments
{count} votes

Accepted answer
  1. Wallin, Niklas 96 Reputation points
    2021-02-12T10:47:09.5+00:00

    I found a solution to my problem using the "externaldata" function. I created a CSV file with the Instance ID's and some other parameters as my reference and then uploaded it to a blob storage. Then I joined the result from the external data function with the log data produced by the applications with a left-outer join. In that case all the Instance ID's would show up regardless if those instances have reported any data. Then I could write queries to act on the missing data. So:

    externaldata(Instance:string, CustomerName:string) [@"<link to static CSV file"];
    | join kind = leftouter (
    MyCustomLogTable
    | <query filters >
    ) on $left.Instance == $right.InstanceID
    | project <result params>
    
    0 comments No comments

0 additional answers

Sort by: Most helpful