How to change logLevel for velero in AKS backup

Question

When you install the AKS backup extension it sets up velero with logLevel=debug. This generates a huge amount of logs (by far the biggest on our cluster). I want to only allow velero to log warning, but if I try and edit the deployment yaml for dataprotection-microsoft-kubernetes-agent it gets set back to debug.

I was hoping to be able to update it via something like this:

az k8s-extension update \
  --cluster-name <cluster-name> \
  --resource-group <resource-group> \
  --cluster-type managedClusters \
  --name azure-aks-backup \
  --configuration-settings "configuration.logLevel=debug"

but this fails with Ignoring unrecognized configuration setting: configuration.logLevel

Answer 1

Ian Cox

Container logs (ContainerLogV2) are usually the top cost driver. This must be done from the Azure Monitor agent ConfigMap, not just Insights settings. You can exclude entire namespaces from log collection.

For example ConfigMap snippet:

[log_collection_settings]
[log_collection_settings.stdout]
enabled = true
exclude_namespaces = ["kube-system", "velero", "gatekeeper-system"]
[log_collection_settings.stderr]
enabled = true
exclude_namespaces = ["kube-system", "velero", "gatekeeper-system"]

Filter container log collection with ConfigMap - https://docs.azure.cn/en-us/azure-monitor/containers/container-insights-data-collection-filter

If you’re still on legacy ContainerLog, you’re paying more than needed.

Key benefits of ContainerLogV2:

1. Supports Basic Logs (lower cost)
2. Better metadata, fewer joins
3. Default for newer clusters / AMA‑based onboarding

Supported and recommended by Microsoft - https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-logs-schema

f you cannot exclude an entire namespace, you can exclude specific workloads.

Add this annotation to the pod/deployment:

annotations:
  fluentbit.io/exclude: "true"

This prevents both stdout & stderr collection for that pod

You have mentioned “by far the most expensive log” this matches Velero running with logLevel=debug, which is a known issue. When installed via AKS Backup Extension, Velero is forced to debug and manual changes get reverted. Produces massive log volume

This is a known limitation, not a misconfiguration - https://stackoverflow.com/questions/73195665/aks-configured-container-insights-does-capture-excluded-namespaces

Post excluding dataprotection-microsoft from stdout in the configmap ,it's working fine.