Edit

Monitor and query Azure Kubernetes Service (AKS) apiserver requests

  • Article

This article describes how to use Azure Diagnostics to enable logging for kube-audit events generated by user and application activities by the kube-apiserver component. Audit events written to the Kubernetes Audit backend are collected and forwarded to your Log Analytics workspace, where you can integrate them into queries, alerts, and visualizations with existing log data.

Before you begin

Collect Kubernetes audit logs

Kubernetes audit logging isn't enabled by default on an AKS cluster on account of Microsoft manages the AKS control plane. You can create diagnostic settings for your cluster resource using any one of the multiple methods described in the Create diagnostic settings article. While configuring diagnostic settings, specify the following:

  • Logs and metrics to route: For logs, choose the category Kubernetes Audit to send to the destination specified later.
  • Destination details: Select the checkbox for Log Analytics.

Note

There could be substantial cost involved once kube-audit logs are enabled. Consider disabling kube-audit logging when not required. An alternative approach to significantly reduce the number of logs and help reduce cost is by enabling collection from kube-audit-admin, which excludes the get and list audit events. For strategies to reduce your Azure Monitor costs, see Cost optimization and Azure Monitor.

After a few moments, the new setting appears in your list of settings for this resource. Logs are streamed to the specified destinations as new event data is generated. It might take up to 15 minutes between when an event is emitted and when it appears in a Log Analytics workspace.

After creating the diagnostic setting to collect kube-audit events, the data can be queried from the AzureDiagnostics table.

Query the apiserver requests

It's often useful to build queries that start with an example or two and then modify them to fit your requirements. To help build more advanced queries, you can experiment with the following sample query.

let starttime = datetime("2023-02-23");
let endtime = datetime("2023-02-24");
AzureDiagnostics
| where TimeGenerated between(starttime..endtime)
| where Category == "kube-audit"
| extend event = parse_json(log_s)
| extend HttpMethod = tostring(event.verb)
| extend User = tostring(event.user.username)
| extend Apiserver = pod_s
| extend SourceIP = tostring(event.sourceIPs[0])
| project TimeGenerated, Category, HttpMethod, User, Apiserver, SourceIP, OperationName, event

Next steps

For more information about AKS metrics, logs, and other important values, see Monitoring AKS data reference.