How to not duplicate data to LA for VM AMA via Sentinel vs Defender vs VM Insights vs Policy

Question

In order to get VM related logs we need Azure Monitor Agent (AMA) to be installed on VM's, in order to get data from VM's to Log Analytics (LA) workspace we need Data Collection Rule (DCR). Our question is that

1. We can use Sentinel Data connector for AMA

• This will automatically install the agent and bring 'Security' related logs via DCR to LA

1. We can use Azure Defender for Cloud (Servers option) to install AMA and specify LA - the DCR is created automatically using this option
2. We can use Azure policy to install AMA and create DCR to land data in LA
3. We can also enable VM insights which creates its own custom DCR and specify LA

Our Question is which one to use that doesnt duplicate the data ??

We want to use VM insights to get data on VM but its creating its own DCR and landing in the same LA , we want to enforce installing AMA on all VM via policy again which needs its own set of DCR , and we also want to use Azure defender features for Server which creates its own DCR and lastly Sentinel creates its own DCR and lands in same LA (albeit it only collects security related events). So all these DCR's will be pushing the same data to the same LA via different venues which we dont have any control or any way to reference using the same DCR.

So which one to use without writing the same data atleast 4 times to the same LA.

Answer 1

Hi,

It is up to you if you want to consolidate everything in one DCR or multiple ones. My advice is to have different DCRs for different types of data. That way if you do not want to pull certain data for certain machines you can just not assign that DCR. Of course, the DCRs should have different data to collect specified to avoid duplicate data pushed to the same Log Analytics workspace. If you give me the DCR definition for each of these I can give particular advice but overall, you are not obligated to use the built-in Azure Policies. If they do not suit your needs, you can just duplicate them and modify them according to your needs.

Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

Answer 2

Hi Akhan,

I have some experience on this which may contribute some insight.

Generally speaking it shouldn't matter if you create your DCR inside Sentinel or just from the DCR blade or within Defender for Cloud. i.e. this should not result in duplicate data.

Also, DCR seems to be pretty good about not allowing duplicates, so if you try to create 2 DCRs for the same scope it 'should' be greyed out.

Creating separate DCRs for each unique collection method (D4Cloud, Insights, etc) is advised and sometimes necessary.

Collecting syslog data can result in duplicates for syslog and CEF specifically, so using specific syslog facilities for each is advisable:
https://techcommunity.microsoft.com/t5/microsoft-sentinel/issue-regarding-logs-duplication-in-cef-via-ama-method/m-p/3695791

Unlrelated to duplicate data:

Note that if you need CEF syslog data you should do it in Sentinel because it will push down a specific configuration to the AMA agent.