Not able to collect custom text logs in azure log analytics workspace using Data Collection rules

Question

Hi, I am facing one issue where i am not able to collect custom application text logs in azure log analytics workspace using Data Collection rules.

The following steps i did. 1. Created Data collection Endpoint in the same region.

2. Created Custom table in log analytics workspace. 3. Created Data collection Rule in the Same region by selecting custom text logs and defining the log path and the above created table and attached the application VM in the resources as well. 4.After that, i tried to query the log analytics table, i can only see heartbeat metrics and custom logs were not visible. Can you anyone help me or guide me if i did anything wrong on the above steps.

Answer 1

Check the metrics such as Logs Ingestion Bytes per Min and Logs Rows Received per Min to ensure that data is reaching Azure Monitor. If these metrics show activity, it means data is being collected but might not be appearing in the table due to some configuration issue.

https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/data-collection-monitor

https://mortenknudsen.net/?p=1453

Answer 2

Thanks Vahid.
Will check the suggestions and the links provided.

Answer 3

Hi hajo

Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

Make sure the Azure Monitor Agent can read the log files. Right-click the file, go to Properties > Security, and check that the agent has permission to read the file.

Ensure your log files are in the right format. Each entry should be on one line or have a timestamp at the start (e.g., YYYY-MM-DD HH:MM:SS). Otherwise, they may not be processed correctly.

Make sure your custom table in Log Analytics is set up correctly to receive log data. Check that the schema matches the data being collected.

Check that the Data Collection Endpoint (DCE) is properly created and linked to your DCR. The DCE must be in the same region as your Log Analytics workspace.

Ensure the Azure Monitor Agent is running on your virtual machine. You can check this in the Services console or by using PowerShell commands.

After setup, it may take some time for data to appear in Log Analytics, usually up to an hour.

For reference, please review this documentation:-
https://learn.microsoft.com/en-us/azure/azure-monitor/agents/data-sources-custom-logs?toc=%2Fazure%2Fazure-monitor%2Ftoc.json
https://learn.microsoft.com/th-th/azure/azure-monitor/agents/data-collection-log-text?tabs=portal

If you have any further queries, do let us know.

---

If the answer is helpful, please click "Accept Answer" and "Upvote it".