How/Is it possible to filter in/out logs from AKS on Log Analytics

Question

How/Is it possible to filter in/out logs from AKS on Log Analytics

Thomas Ferfelis 0

Hello,

I want to ask how and whether is it possible to filter logs that are coming from AKS on Log Analytics workspace. In our case we want to exclude some containers from the sending logs on log analytics with the purpose of reducing the amount of unneeded logs on workspace.

Thanks,

Thomas

Anonymous

2024-07-31T11:08:15.3233333+00:00
Hi Thomas,

Thank you for reaching out to us on the Microsoft Q&A forum.

Yes, it is possible to filter logs from specific containers in AKS on a Log Analytics workspace. To achieve this, you can use the where clause in your Kusto query to filter logs based on the container name.

Here's an example query that filters logs from a container named excluded-container:

kustoVerifyOpen In EditorEditCopy code1ContainerInventory2| where ContainerName != "excluded-container"3| project Computer, Name, Image, ImageTag, ContainerState, CreatedTime, StartedTime, FinishedTime4| render table

Example, where clause filters the logs where the ContainerName is not equal to excluded-container. You can replace excluded-container with the name of the container you want to exclude.

To filter logs from multiple containers, you can use the not in operator:

kustoVerifyOpen In EditorEditCopy code1ContainerInventory2| where ContainerName !in ("excluded-container-1", "excluded-container-2")3| project Computer, Name, Image, ImageTag, ContainerState, CreatedTime, StartedTime, FinishedTime4| render table

In this example, the where clause filters the logs where the ContainerName is not in the list ("excluded-container-1", "excluded-container-2").

Please find the attached document for your reference: Query logs from Container insights.

If you have any further questions, please don't hesitate to reach us.

Regards.
Thomas Ferfelis 0 Reputation points

2024-07-31T11:13:39.8233333+00:00

Hello,

What i want to find a way is to filter the log collection and not the logs that queries returns. So they don't reach the tables in the first place. So to reduce the total amount of logs collected.

Thanks,

Thomas
Anonymous

2024-08-01T11:36:21.2833333+00:00
Hi Thomas,
Please create one more configmaps in cluster with settings of logs indexation/exclusions. You can add configuration block or settings as per your requirement.
The configuration change can take a few minutes to finish before taking effect.

Reference check it in provided links:

Configure and deploy ConfigMap

2.Container insights log schem

If you have any further queries, do let us know.

Thank you.
Anonymous

2024-08-02T13:45:04.49+00:00

Hi Thomas,

Just checking in to see if you had a chance to see my response to your question.

Please tell us if it was helpful and feel free to reach out to us if you have any queries.

Thank you.
Thomas Ferfelis 0 Reputation points

2024-08-05T06:45:25.7633333+00:00

Hello,

Yeah, thank you for the response. It is actually what we are interested in. But i see some limitations on exclusions. We were more interested in excluding specific containers for log collection but seems that the exclusion part is applicable on namespace level only.

Thanks,

Thomas
Thomas Ferfelis 0 Reputation points

2024-08-05T06:48:21.17+00:00

Hello,

Yeah, thank you for the response. It is actually what we are interested in. But i see some limitations on namespace exclusions. It seems that the exclusion can happen only on namespace level and not on container level which in our case it would work only if we group our non-logging-interested apps on a specific ns and exclude it from log collection itself. Any "popular" workarounds on that issue?

Thanks,

Thomas

Kasun Rajapakse 431

Hello Thomas,

Did you check this configuration in the above shared config map

[log_collection_settings.stderr]
          # Default value for enabled is true
          enabled = true
          # exclude_namespaces setting holds good only if enabled is set to true
          # kube-system,gatekeeper-system log collection are disabled by default in the absence of 'log_collection_settings.stderr' setting. If you want to enable kube-system,gatekeeper-system, remove them from the following setting.
          # If you want to continue to disable kube-system,gatekeeper-system log collection keep the namespaces in the following setting and add any other namespace you want to disable log collection to the array.
          # In the absense of this configmap, default value for exclude_namespaces = ["kube-system","gatekeeper-system"]
          exclude_namespaces = ["kube-system","gatekeeper-system"]
          # If you want to collect logs from only selective pods inside system namespaces add them to the following setting. Provide namepace:controllerName of the system pod. NOTE: this setting is only for pods in system namespaces
          # Valid values for system namespaces are: kube-system, azure-arc, gatekeeper-system, kube-public, kube-node-lease, calico-system. The system namespace used should not be present in exclude_namespaces
          # collect_system_pod_logs = ["kube-system:coredns"]

In here, we can exclude namespaces and pods in specific namespace. Please refer the last setting

Kasun Rajapakse 431

Helo Thomas,

Did you check the below configuration in the configmap shared above

[log_collection_settings.stderr]
          # Default value for enabled is true
          enabled = true
          # exclude_namespaces setting holds good only if enabled is set to true
          # kube-system,gatekeeper-system log collection are disabled by default in the absence of 'log_collection_settings.stderr' setting. If you want to enable kube-system,gatekeeper-system, remove them from the following setting.
          # If you want to continue to disable kube-system,gatekeeper-system log collection keep the namespaces in the following setting and add any other namespace you want to disable log collection to the array.
          # In the absense of this configmap, default value for exclude_namespaces = ["kube-system","gatekeeper-system"]
          exclude_namespaces = ["kube-system","gatekeeper-system"]
          # If you want to collect logs from only selective pods inside system namespaces add them to the following setting. Provide namepace:controllerName of the system pod. NOTE: this setting is only for pods in system namespaces
          # Valid values for system namespaces are: kube-system, azure-arc, gatekeeper-system, kube-public, kube-node-lease, calico-system. The system namespace used should not be present in exclude_namespaces
          # collect_system_pod_logs = ["kube-system:coredns"]

In this section you can exclude namespaces or pods in a namespace for log collection

1 answer

Your answer

Anonymous

2024-07-31T11:08:15.3233333+00:00

Hi Thomas,

Thank you for reaching out to us on the Microsoft Q&A forum.

Yes, it is possible to filter logs from specific containers in AKS on a Log Analytics workspace. To achieve this, you can use the where clause in your Kusto query to filter logs based on the container name.

Here's an example query that filters logs from a container named excluded-container:

kustoVerifyOpen In EditorEditCopy code1ContainerInventory2| where ContainerName != "excluded-container"3| project Computer, Name, Image, ImageTag, ContainerState, CreatedTime, StartedTime, FinishedTime4| render table

Example, where clause filters the logs where the ContainerName is not equal to excluded-container. You can replace excluded-container with the name of the container you want to exclude.

To filter logs from multiple containers, you can use the not in operator:

kustoVerifyOpen In EditorEditCopy code1ContainerInventory2| where ContainerName !in ("excluded-container-1", "excluded-container-2")3| project Computer, Name, Image, ImageTag, ContainerState, CreatedTime, StartedTime, FinishedTime4| render table

In this example, the where clause filters the logs where the ContainerName is not in the list ("excluded-container-1", "excluded-container-2").

Please find the attached document for your reference: Query logs from Container insights.

If you have any further questions, please don't hesitate to reach us.

Regards.
Thomas Ferfelis 0 Reputation points

2024-07-31T11:13:39.8233333+00:00

Hello,

What i want to find a way is to filter the log collection and not the logs that queries returns. So they don't reach the tables in the first place. So to reduce the total amount of logs collected.

Thanks,

Thomas
Anonymous

2024-08-01T11:36:21.2833333+00:00

Hi Thomas,
Please create one more configmaps in cluster with settings of logs indexation/exclusions. You can add configuration block or settings as per your requirement.
The configuration change can take a few minutes to finish before taking effect.

Reference check it in provided links:

Configure and deploy ConfigMap

2.Container insights log schem

If you have any further queries, do let us know.

Thank you.
Anonymous

2024-08-02T13:45:04.49+00:00

Hi Thomas,

Just checking in to see if you had a chance to see my response to your question.

Please tell us if it was helpful and feel free to reach out to us if you have any queries.

Thank you.
Thomas Ferfelis 0 Reputation points

2024-08-05T06:45:25.7633333+00:00

Hello,

Yeah, thank you for the response. It is actually what we are interested in. But i see some limitations on exclusions. We were more interested in excluding specific containers for log collection but seems that the exclusion part is applicable on namespace level only.

Thanks,

Thomas
Thomas Ferfelis 0 Reputation points

2024-08-05T06:48:21.17+00:00

Hello,

Yeah, thank you for the response. It is actually what we are interested in. But i see some limitations on namespace exclusions. It seems that the exclusion can happen only on namespace level and not on container level which in our case it would work only if we group our non-logging-interested apps on a specific ns and exclude it from log collection itself. Any "popular" workarounds on that issue?

Thanks,

Thomas
Kasun Rajapakse 431 Reputation points

2024-08-05T18:38:12.75+00:00

Hello Thomas,

Did you check this configuration in the above shared config map

[log_collection_settings.stderr] # Default value for enabled is true enabled = true # exclude_namespaces setting holds good only if enabled is set to true # kube-system,gatekeeper-system log collection are disabled by default in the absence of 'log_collection_settings.stderr' setting. If you want to enable kube-system,gatekeeper-system, remove them from the following setting. # If you want to continue to disable kube-system,gatekeeper-system log collection keep the namespaces in the following setting and add any other namespace you want to disable log collection to the array. # In the absense of this configmap, default value for exclude_namespaces = ["kube-system","gatekeeper-system"] exclude_namespaces = ["kube-system","gatekeeper-system"] # If you want to collect logs from only selective pods inside system namespaces add them to the following setting. Provide namepace:controllerName of the system pod. NOTE: this setting is only for pods in system namespaces # Valid values for system namespaces are: kube-system, azure-arc, gatekeeper-system, kube-public, kube-node-lease, calico-system. The system namespace used should not be present in exclude_namespaces # collect_system_pod_logs = ["kube-system:coredns"]

In here, we can exclude namespaces and pods in specific namespace. Please refer the last setting
Kasun Rajapakse 431 Reputation points

2024-08-05T18:41:10.02+00:00

Helo Thomas,

Did you check the below configuration in the configmap shared above

[log_collection_settings.stderr] # Default value for enabled is true enabled = true # exclude_namespaces setting holds good only if enabled is set to true # kube-system,gatekeeper-system log collection are disabled by default in the absence of 'log_collection_settings.stderr' setting. If you want to enable kube-system,gatekeeper-system, remove them from the following setting. # If you want to continue to disable kube-system,gatekeeper-system log collection keep the namespaces in the following setting and add any other namespace you want to disable log collection to the array. # In the absense of this configmap, default value for exclude_namespaces = ["kube-system","gatekeeper-system"] exclude_namespaces = ["kube-system","gatekeeper-system"] # If you want to collect logs from only selective pods inside system namespaces add them to the following setting. Provide namepace:controllerName of the system pod. NOTE: this setting is only for pods in system namespaces # Valid values for system namespaces are: kube-system, azure-arc, gatekeeper-system, kube-public, kube-node-lease, calico-system. The system namespace used should not be present in exclude_namespaces # collect_system_pod_logs = ["kube-system:coredns"]

In this section you can exclude namespaces or pods in a namespace for log collection

Answer 1

Thomas Ferfelis 0

Hello,

Yeah but if it comes to exclude pods in a non-system ns you cant with this configuration. Although we did find a way to exclude using this configmap by annotating pods for exclusion. Your answer was helpful enough for us.

Thanks,

Thomas

#[log_collection_settings.filter_using_annotations]
# if enabled will exclude logs from pods with annotations fluenbit.io/exclude: "true".
# Read more: https://docs.fluentbit.io/manual/pipeline/filters/kubernetes#kubernetes-annotations
	# enabled = false

Anonymous

2024-08-07T08:04:59.9433333+00:00

Hi Thomas Ferfelis,

we are glad to hear that your issue has been resolved.

If you have any questions in future, please don’t hesitate to ask. We’re glad to help you.

Thank You.
Hannanel Gershinsky 0 Reputation points

2025-05-13T11:01:29.9033333+00:00

is there a way to do this without needing to annotate every containers deployment? i have a microservice application that deploys about 50 services, and only need the logs from one.

Share via

How/Is it possible to filter in/out logs from AKS on Log Analytics

1 answer

Your answer