Resolving Duplicate Data Issues on Azure VM

Question

What could cause my Azure VM to accumulate duplicate data, and what steps can I take to fix it?

PS - Based on common issues that we have seen from customers and other sources, we are posting these questions to help the Azure community.

Answer 1

If your Azure VM is accumulating duplicate data when the Azure Monitor Agent (AMA) is installed, the issue could be related to the configuration of the agent or how it interacts with your VM and the Azure environment. Here are some potential causes :

• Creating multiple DCRs with the same data source and associating them to the same agent. Ensure that you're filtering data in the DCRs such that each collects unique data.
• Creating a DCR that collects security logs and enabling Sentinel for the same agents. In this case, you may collect the same events in the Event table and the SecurityEvent table.
• Using both the Azure Monitor agent and the legacy Log Analytics agent on the same machine. Limit duplicate events to only the time when you transition from one agent to the other.
• Using the Azure Monitor agent and the Azure Diagnostics extension on the same machine. This can cause duplicate data collection.
• The same VM might be mapped to multiple Log Analytics workspaces, resulting in data being sent to multiple locations and causing duplicates.

After performing these actions, check your VM to confirm that the data accumulation problem has been resolved.

Resources:

• Azure Monitor Agent overview
• Manage extensions for a VM

**Please do not forget to "Accept the answer" and "up-vote" wherever the information provided helps you, as this can be beneficial to other community members.**If your Azure VM is accumulating duplicate data when the Azure Monitor Agent (AMA) is installed, the issue could be related to the configuration of the agent or how it interacts with your VM and the Azure environment. Here are some potential causes :

• Creating multiple DCRs with the same data source and associating them to the same agent. Ensure that you're filtering data in the DCRs such that each collects unique data.
• Creating a DCR that collects security logs and enabling Sentinel for the same agents. In this case, you may collect the same events in the Event table and the SecurityEvent table.
• Using both the Azure Monitor agent and the legacy Log Analytics agent on the same machine. Limit duplicate events to only the time when you transition from one agent to the other.
• Using the Azure Monitor agent and the Azure Diagnostics extension on the same machine. This can cause duplicate data collection.
• The same VM might be mapped to multiple Log Analytics workspaces, resulting in data being sent to multiple locations and causing duplicates.

After performing these actions, check your VM to confirm that the data accumulation problem has been resolved.

Resources:

• Azure Monitor Agent overview
• Manage extensions for a VM

Please do not forget to "Accept the answer" and "up-vote" wherever the information provided helps you, as this can be beneficial to other community members.