i want to create Alert for AKS nodes.

Hi Anjali Gobari,

I understand that you want to set up alerts to your AKS nodes using Log Analytics and Kusto Query Language. These metrics can be found in the KubeNodeInventory tables in your Log Analytics workspace. This Table stores Kubernetes cluster's node information.

Below is the sample Kusto query for MemoryPressure and can be used for other metrics as well.

KubeNodeInventory | where TimeGenerated > ago(15m) | where Condition == "MemoryPressure" and Status == "true" | project TimeGenerated, ClusterName, Node, Condition, Status

This query helps identify Kubernetes nodes that are experiencing Memory Pressure within the last 15 minutes.

To create an alert for it, go to log analytics workspace and to Logs and run the query then create an alert for it.

Please refer to the document about the kubenodeinventory

https://learn.microsoft.com/en-us/azure/azure-monitor/reference/tables/kubenodeinventory

If you have any further queries, let me know. If the information is helpful, please click on Upvote.

Thank you

Hi Dharani,

Thanks for the swift response, however i dont see the "Condition" coloumn in the KubeNodeInventory table. Checked in the Log analytics workspace and reference as well.

is there something i am missing?

Thanks

Anjali

Hi Anjali Gobari,

Node Memory pressure is the level of stress on available node memory resources within a network. It measures how close a node is to reaching its available memory.

I request you to please try the query below to check the avg node memory usage percentage per 5min over the last hour and can set alert based on your preferred threshold.

let endDateTime = now();
let startDateTime = ago(1h);
let trendBinSize = 5m;
let capacityCounterName = 'memoryCapacityBytes';
let usageCounterName = 'memoryRssBytes';
KubeNodeInventory
| where TimeGenerated < endDateTime
| where TimeGenerated >= startDateTime
// cluster filter would go here if multiple clusters are reporting to the same Log Analytics workspace
| distinct ClusterName, Computer, _ResourceId
| join hint.strategy=shuffle (
  Perf
  | where TimeGenerated < endDateTime
  | where TimeGenerated >= startDateTime
  | where ObjectName == 'K8SNode'
  | where CounterName == capacityCounterName
  | summarize LimitValue = max(CounterValue) by Computer, CounterName, bin(TimeGenerated, trendBinSize)
  | project Computer, CapacityStartTime = TimeGenerated, CapacityEndTime = TimeGenerated + trendBinSize, LimitValue
) on Computer
| join kind=inner hint.strategy=shuffle (
  Perf
  | where TimeGenerated < endDateTime + trendBinSize
  | where TimeGenerated >= startDateTime - trendBinSize
  | where ObjectName == 'K8SNode'
  | where CounterName == usageCounterName
  | project Computer, UsageValue = CounterValue, TimeGenerated
) on Computer
| where TimeGenerated >= CapacityStartTime and TimeGenerated < CapacityEndTime
| project ClusterName, Computer, TimeGenerated, UsagePercent = UsageValue * 100.0 / LimitValue, _ResourceId
| summarize AggregatedValue = avg(UsagePercent) by bin(TimeGenerated, trendBinSize), ClusterName, _ResourceId

Also, from the Cluster Alerts Wizard, we can set up the alerts for Node CPU usage percentage is greater than the threshold that we set. Please refer Alert rules for cluster.

Please let me know if you have questions.

Hi Dharani,

That is really informative and helpful.

Node memory and disk utilization is something I can see the metric in perf table or azure diagnostics. What I am actually concerned about is the PID pressure which I am not finding in the log Analytics tables. I was assuming there will be direct metrics for these.

Please help me in understanding how can Node PID pressure be calculated?

Thanks