AKS Container Insights - logs configuration/logs filter

Elena Fomina 1 Reputation point
2021-09-29T15:12:14.237+00:00

Hello there,

I was wondering if it it possible to somehow configure which logs are sent to Azure Monitor when Container Insight solution is turned on?
How I imagine it, is that I would list the metrics/certain logs that I would like to see in Monitor and the rest of the logs or metrics will not be even sent.
In the end I would like to optimise my Log Analytics costs by reducing the amount of data that are sent to workspace.

I was looking at omsagent-rs-config ConfigMap but it looks like it doesn't really work.

Any ideas please? :)

Azure Monitor
Azure Monitor
An Azure service that is used to collect, analyze, and act on telemetry data from Azure and on-premises environments.
2,820 questions
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
1,876 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. SRIJIT-BOSE-MSFT 4,331 Reputation points Microsoft Employee
    2021-10-01T22:38:31.297+00:00

    @Elena Fomina , Thank you for your question.

    If you do a kubectl describe cm omsagent-rs-config -n kube-system you shall find an interesting label: addonmanager.kubernetes.io/mode=Reconcile

    As discussed here,

    Addons with label addonmanager.kubernetes.io/mode=Reconcile will be periodically reconciled. Direct manipulation to these addons through apiserver is discouraged because addon-manager will bring them back to the original state. In particular:

    • Addon will be re-created if it is deleted.
    • Addon will be reconfigured to the state given by the supplied fields in the template file periodically.
    • Addon will be deleted when its manifest file is deleted from the $ADDON_PATH.

    The $ADDON_PATH by default is set to /etc/kubernetes/addons/ on the control plane node(s).

    For more information please check this document.

    Since AKS is a managed Kubernetes Service you will not be able to access $ADDON_PATH. We strongly recommend against forcing changes to kube-system resources as these are critical for the proper functioning of the cluster.

    ----

    [ Very Important: Please remember that the downloadable Configmap YAML shared in both the instruction links are the same. You have to download it once and update both settings in the same YAML manifest. Then perform kubectl apply -f <configmap_yaml_file.yaml> where configmap_yaml_file.yaml contains the modified configurations for both Container Insights data and Metrics collection as well as Prometheus scraping.]

    The configuration change can take a few minutes to finish before taking effect, and all omsagent pods in the cluster will restart. The restart is a rolling restart for all omsagent pods, not all restart at the same time. When the restarts are finished, a message is displayed that's similar to the following and includes the result: configmap "container-azm-ms-agentconfig" created.

    To verify the configuration was successfully applied to a cluster please follow the instructions here.

    ----
    Hope this helps.

    Please "Accept as Answer" if it helped, so that it can help others in the community looking for help on similar topics.

    1 person found this answer helpful.
    0 comments