Microsoft AMA does not send logs to Log Analytics Workspace

Question

Hi all,

There is a RHEL server with 2 interfaces. logs coming to these to interfaces can be written to a file ( via rsyslog , "omfile" ). However, logs coming to newly defined interface cannot be written to sentinel logs. The error that i only can see is this:

2024-05-07T11:49:55.0626210Z: Error while inserting item to Local persistent store local7.notice: IO error: No space left on device: While open a file for appending: /var/opt/microsoft/azuremonitoragent/events/local7.notice/677638.log: No space left on device

But there is no any disk problem or open process problem. I already reviewed "Syslog troubleshooting guide for Azure Monitor Agent for Linux" article.

Do you have any recommendation about this issue?

Answer 1

Hello,
Thank you for your question.
Based on the syslog troubleshooting guide the Error you are experiencing is mentioned and the below summary covers it:
The issue is caused by a full disk. The default configuration of rsyslog logs all facilities to /var/log, which can lead to the disk filling up quickly.

Kindly Confirm the Issue: To confirm that the disk is full, you can check the available disk space using the df -h command. This will show you the disk usage and help you identify if the disk is indeed full.

Fix: To fix this issue, you need to remove high-volume facilities from the rsyslog configuration file. Specifically, you should edit the /etc/rsyslog.d/50-default.conf file and remove or comment out the lines that log high-volume facilities to /var/log.

You mentioned that you have looked at the Syslog troubleshooting guide for Azure Monitor Agent for Linux Doc but kindly, double check with the above recommendation since it is the direction provided to fix the issue. Doc Ref Syslog troubleshooting guide for Azure Monitor Agent for Linux.