Data collection rules in Azure Monitor

Article
03/19/2023

Data collection rules (DCRs) define the data collection process in Azure Monitor. DCRs specify what data should be collected, how to transform that data, and where to send that data. Some DCRs will be created and managed by Azure Monitor to collect a specific set of data to enable insights and visualizations. You might also create your own DCRs to define the set of data required for other scenarios.

View data collection rules

To view your DCRs in the Azure portal, select Data Collection Rules under Settings on the Monitor menu.

Note

Although this view shows all DCRs in the specified subscriptions, selecting the Create button will create a data collection for Azure Monitor Agent. Similarly, this page will only allow you to modify DCRs for Azure Monitor Agent. For guidance on how to create and update DCRs for other workflows, see Create a data collection rule.

Create a data collection rule

The following resources describe different scenarios for creating DCRs. In some cases, the DCR might be created for you. In other cases, you might need to create and edit it yourself.

Scenario	Resources	Description
Azure Monitor Agent	Configure data collection for Azure Monitor Agent	Use the Azure portal to create a DCR that specifies events and performance counters to collect from a machine with Azure Monitor Agent. Then apply that rule to one or more virtual machines. Azure Monitor Agent will be installed on any machines that don't currently have it.
	Use Azure Policy to install Azure Monitor Agent and associate with a DCR	Use Azure Policy to install Azure Monitor Agent and associate one or more DCRs with any virtual machines or virtual machine scale sets as they're created in your subscription.
Custom logs	Configure custom logs by using the Azure portal Configure custom logs by using Azure Resource Manager templates and the REST API	Send custom data by using a REST API. The API call connects to a data collection endpoint and specifies a DCR to use. The DCR specifies the target table and potentially includes a transformation that filters and modifies the data before it's stored in a Log Analytics workspace.
Workspace transformation	Configure ingestion-time transformations by using the Azure portal Configure ingestion-time transformations by using Azure Resource Manager templates and the REST API	Create a transformation for any supported table in a Log Analytics workspace. The transformation is defined in a DCR that's then associated with the workspace. It's applied to any data sent to that table from a legacy workload that doesn't use a DCR.

Work with data collection rules

To work with DCRs outside of the Azure portal, see the following resources:

Method	Resources
API	Directly edit the DCR in any JSON editor and then install it by using the REST API.
CLI	Create DCRs and associations with the Azure CLI.
PowerShell	Work with DCRs and associations with the following Azure PowerShell cmdlets: Get-AzDataCollectionRule New-AzDataCollectionRule Set-AzDataCollectionRule Update-AzDataCollectionRule Remove-AzDataCollectionRule Get-AzDataCollectionRuleAssociation New-AzDataCollectionRuleAssociation Remove-AzDataCollectionRuleAssociation

Structure of a data collection rule

Data collection rules are formatted in JSON. Although you might not need to interact with them directly, there are scenarios where you might need to directly edit a DCR. For a description of this structure and the different elements used for different workflows, see Data collection rule structure.

Permissions

When you use programmatic methods to create DCRs and associations, you require the following permissions:

Built-in role	Scopes	Reason
Monitoring Contributor	Subscription and/or Resource group and/or An existing DCR	Create or edit DCRs, assign rules to the machine, deploy associations ).
Virtual Machine Contributor Azure Connected Machine Resource Administrator	Virtual machines, virtual machine scale sets Azure Arc-enabled servers	Deploy agent extensions on the VM.
Any role that includes the action Microsoft.Resources/deployments/*	Subscription and/or Resource group and/or An existing DCR	Deploy Azure Resource Manager templates.

Limits

For limits that apply to each DCR, see Azure Monitor service limits.

Supported regions

Data collection rules are available in all public regions where Log Analytics workspaces and the Azure Government and China clouds are supported. Air-gapped clouds aren't yet supported.

Single region data residency is a preview feature to enable storing customer data in a single region and is currently only available in the Southeast Asia Region (Singapore) of the Asia Pacific Geo and the Brazil South (Sao Paulo State) Region of the Brazil Geo. Single-region residency is enabled by default in these regions.

Data resiliency and high availability

A rule gets created and stored in a particular region and is backed up to the paired-region within the same geography. The service is deployed to all three availability zones within the region. For this reason, it's a zone-redundant service, which further increases availability.