Azure aks monitor alerts triggered only once not every 5 minutes

Bala Krishna 101 Reputation points
2021-10-20T09:53:23.147+00:00

I have a aks cluster in Azure . I wrote query to fetch "Crashloop Backoff" pods every five minutes but this will trigger alert only once not every 5 minutes

KubeEvents
| where ClusterName =~ 'my-aks-test-01'
| where ObjectKind =~ 'Pod'
| where Reason =~ 'BackOff'
| project TimeGenerated, Count,Name,Reason
| summarize AggregatedValue=sum(Count) by bin(TimeGenerated, 1m)
| where todouble(AggregatedValue) > 1

142014-image.png

Azure Monitor
Azure Monitor
An Azure service that is used to collect, analyze, and act on telemetry data from Azure and on-premises environments.
2,792 questions
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
1,852 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Stanislav Zhelyazkov 20,941 Reputation points MVP
    2021-10-21T13:32:14.083+00:00

    Hi,
    It seems you are using Log Alert v2. These alerts have auto mitigate option (stateful alerts) which will trigger once in Fired state once the threshold is met and will trigger another time with Resolved state once threshold is no longer met. Auto mitigate option is useful if you have logs like metrics / performance data that have certain frequency of the data being logged. In your case it is better to edit the alert and uncheck auto mitigate option.

    Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

    0 comments No comments