Hi all, I am sorry, probably, I have stupid question but after my testing to implement alert rules I am confused. I try to configure some alerts for Azure VMs and Azure ARC (for heartbeat, CPU, Memory and so on) I have found this article https://learn.microsoft.com/en-us/azure/azure-monitor/vm/monitor-virtual-machine-alerts#log-alert-rules-1 how to create a query and it shows me some outputs in Logs. But measurement and alert logic are confused me and I don't get any alerts. If someone could show me what to set exactly in a next configuration it would be great! I have implemented alerts of heartbeat for Azure VMs using this link https://learn.microsoft.com/en-us/azure/azure-monitor/vm/tutorial-monitor-vm-alert-availability and it works nice! My screenshots is for ARC machines (haven't found another solution for ARC heartbeating) 2nd question about CPU checking. I need to alert for 100% CPU using during 1h, what query should be and what alert settings? I hope to get help with it and hope to understand logic of alerts TIA, Viktor

To start, try using the following measurements with your customer alert rule. Run the query first in Logs under your workspace to make sure you have records being returned. For Arc heartbeats, those machines connected to the workspace should be sending a heartbeat metric value roughly every minute. For an alert, set the Aggregation Type to Count and Threshold to match the evaluation granularity. An example query for metrics rules is, but honestly the query you have above should work for both. Heartbeat | where TimeGenerated > ago(5m) | summarize AggregatedValue = count() by Computer

Azure alert setting for custom log search

Accepted answer

Ryan Hill 28,106 Reputation points Microsoft Employee

2023-09-26T01:53:37.53+00:00
To start, try using the following measurements with your customer alert rule.

Run the query first in Logs under your workspace to make sure you have records being returned. For Arc heartbeats, those machines connected to the workspace should be sending a heartbeat metric value roughly every minute. For an alert, set the Aggregation Type to Count and Threshold to match the evaluation granularity. An example query for metrics rules is, but honestly the query you have above should work for both.

Heartbeat | where TimeGenerated > ago(5m) | summarize AggregatedValue = count() by Computer
Please sign in to rate this answer.

1 person found this answer helpful.
Viktor Korokhov 130 Reputation points

2023-09-26T07:34:15.05+00:00

Many thanks for answer. I have solved my problem. My query works fine for ARC machines:

Heartbeat | where ResourceType == "machines" | summarize TimeGenerated=max(TimeGenerated) by Computer, _ResourceId | extend Duration = datetime_diff('minute', now(), TimeGenerated) | summarize AggregatedValue = min(Duration) by Computer, bin(TimeGenerated, 5m), _ResourceId | where AggregatedValue > 2

Seems, query output AggregatedValue = 2 periodicaly. I guess it is for network latency.

My alert wasn't fired (and wasn't sent) before because if it is happened once for source alert will not be fired again until previous fired status is closed, even if you create a new rule.

Alert settings for my case are next:

And useful feature "Automatically resolve alerts (preview)" in Details of alert settings.

I hope my case might help to someone else.

Just one more problem for me now - how to suppress alerts during periodic maintenance break (each last full weekend(Sat, Su) of month)?

Ryan Hill 28,106 Reputation points Microsoft Employee

2023-09-26T12:18:42.7666667+00:00

You can exclude weekends in your KQL query using | where datetimepart('weekday', timestamp) not in (0, 6)

Viktor Korokhov 130 Reputation points

2023-09-26T13:50:27.0966667+00:00

Inetresting hint but I guess your example will exlude all Sat and Sun from the alerts. It is not that I need. I think this solution should be fine https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-processing-rules?tabs=portal#suppress-notifications-during-planned-maintenance but again it is hard to set exactly only last Sat and Sun of each month.

Ryan Hill 28,106 Reputation points Microsoft Employee

2023-10-02T15:42:20.3466667+00:00

Not going to lie @Viktor Korokhov excluding the last Sat and Sun of the month isn't straightforward. I'll pass this along to some subject matter experts and see if they have any suggestions.

Viktor Korokhov 130 Reputation points

2023-10-04T09:28:19.91+00:00

Me again here :) I just would like to write here some comments about solution above. Seems, this solution is not good for us because we maintenance break last full Saturday + Sunday of each month according SLA. If you set Automatically resolve alerts (preview) in Details of Alert - it great feauture but it has limit of 2 days. I have found it here https://learn.microsoft.com/en-us/answers/questions/1070720/issue-with-azure-monitor-alerting-heartbeat . So, if your VMs are down as planned more 2 days alert will send recovery notification and move status from Fired to Resolved automatically after 2 days even if it is not true.

If you uncheck Automatically resolve alerts (preview) option and stay all other options as is, notifications will do a spam messages every 5 min during 2 days :) and seems (it is only my thoughts) query doesn't allow you to set Override query time range (it will set 2 days for some reason).

Finally, I have decided to do next configuration:

Query:

Heartbeat | where ResourceType == "machines" | summarize LastHeartbeat=max(TimeGenerated) by Computer | where LastHeartbeat < ago(5m)

Alert settings:

I have increased time range for alert settins from 5 to 15 minutes because AMA starting and ending traffic passing has some delays.

Disadvantage of this solution - manual check of every new alert notification.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Azure alert setting for custom log search

0 additional answers

Your answer