Create and edit data collection rules (DCRs) in Azure Monitor

There are multiple methods for creating a data collection rule (DCR) in Azure Monitor. In some cases, Azure Monitor will create and manage the DCR according to settings that you configure in the Azure portal. In other cases, you might need to create your own DCRs to customize particular scenarios.

This article describes the different methods for creating and editing a DCR. For the contents of the DCR itself, see Structure of a data collection rule in Azure Monitor.

Permissions

You require the following permissions to create DCRs and associations:

Built-in role	Scopes	Reason
Monitoring Contributor	Subscription and/or Resource group and/or An existing DCR	Create or edit DCRs, assign rules to the machine, deploy associations.
Virtual Machine Contributor Azure Connected Machine Resource Administrator	Virtual machines, virtual machine scale sets Azure Arc-enabled servers	Deploy agent extensions on the VM.
Any role that includes the action Microsoft.Resources/deployments/*	Subscription and/or Resource group and/or An existing DCR	Deploy Azure Resource Manager templates.

Automated methods to create a DCR

The following table lists methods to create data collection scenarios using the Azure portal where the DCR is created for you. In these cases you don't need to interact directly with the DCR itself.

Scenario	Resources	Description
Monitor a virtual machine	Enable VM insights overview	When you enable VM insights on a VM, the Azure Monitor agent is installed, and a DCR is created that collects a predefined set of performance counters. You shouldn't modify this DCR.
Container insights	Enable Container insights	When you enable Container insights on a Kubernetes cluster, a containerized version of the Azure Monitor agent is installed, and a DCR is created that collects data according to the configuration you selected. You may need to modify this DCR to add a transformation.
Workspace transformation	Add a transformation in a workspace data collection rule using the Azure portal	Create a transformation for any supported table in a Log Analytics workspace. The transformation is defined in a DCR that's then associated with the workspace. It's applied to any data sent to that table from a legacy workload that doesn't already use a DCR.

Create a DCR

The Azure portal provides a data collection rule wizard for collecting data from virtual machines and for collecting Prometheus metrics from containers.

To create a data collection rule using the Azure CLI, PowerShell, API, or ARM templates, create a JSON file, starting with one of the sample DCRs. Use information in Structure of a data collection rule in Azure Monitor to modify the JSON file for your particular environment and requirements.

Important

Create your data collection rule in the same region as your destination Log Analytics workspace or Azure Monitor workspace. You can associate the data collection rule to machines or containers from any subscription or resource group in the tenant. To send data across tenants, you must first enable Azure Lighthouse.

Portal

On the Monitor menu, select Data Collection Rules > Create to open the page to create a new data collection rule.

Configure the settings in each step of the wizard, as detailed below.

Basics

Screen element	Description
Rule name	Enter a name for the data collection rule.
Subscription	Associate the data collection rule to a subscription.
Resource Group	Associate the data collection rule to a resource group.
Region	Create your data collection rule in the same region as your destination Log Analytics workspace. You can associate the data collection rule to machines from any subscription or resource group in the tenant.
Platform Type	Select Windows or Linux, or All, which allows for both Windows and Linux platforms.
Data Collection Endpoint	To collect Linux syslog data, IIS logs, custom text logs or custom JSON logs, select an existing data collection endpoint or create a new endpoint. You don't need an endpoint to collect performance counters and Windows event logs. On this tab, you can only select a data collection endpoint in the same region as the data collection rule. The agent sends collected data to this data collection endpoint. For more information, see Components of a data collection endpoint.

Resources

Screen element	Description
+ Add resources	Associate virtual machines, Virtual Machine Scale Sets, and Azure Arc for servers to the data collection rule. The Azure portal installs Azure Monitor Agent on resources that don't already have the agent installed.
Enable Data Collection Endpoints	If the machine you're monitoring is not in the same region as your destination Log Analytics workspace, enable data collection endpoints and select an endpoint in the region of the monitored machine to collect Linux syslog data, IIS logs, custom text logs or custom JSON logs. If the monitored machine is in the same region as your destination Log Analytics workspace, or if you're collecting performance counters and Windows event logs, don't select a data collection endpoint on the Resources tab. The data collection endpoint on the Resources tab is the configuration access endpoint, as described in Components of a data collection endpoint. If you need network isolation using private links, select existing endpoints from the same region for the respective resources or create a new endpoint.
Agent extension identity	Use a system-assigned managed identity, or select an existing user-assigned identity assigned to the virtual machine. For more information, see Managed identity types.

Collect and deliver

On the Collect and deliver tab, select Add data source and configure the settings on the Source and Destination tabs, as detailed below.

Screen element	Description
Data source	Select a Data source type and define related fields based on the data source type you select. For more information about collecting data from the various data source types, see Collect data with Azure Monitor Agent
Destination	Add one or more destinations for each source. You can select multiple destinations of the same or different types.

Review + create

Review the data collection rule details and select Create to create the data collection rule.

Note

It can take up to 5 minutes for data to be sent to the destinations when you create a data collection rule using the data collection rule wizard.

CLI

Use the az monitor data-collection rule create command to create a DCR from your JSON file using the Azure CLI as shown in the following example.

az monitor data-collection rule create --location 'eastus' --resource-group 'my-resource-group' --name 'myDCRName' --rule-file 'C:\MyNewDCR.json' --description 'This is my new DCR'

PowerShell

Use the New-AzDataCollectionRule cmdlet to create the DCR from your JSON file using PowerShell as shown in the following example.

New-AzDataCollectionRule -Location 'east-us' -ResourceGroupName 'my-resource-group' -RuleName 'myDCRName' -RuleFile 'C:\MyNewDCR.json' -Description 'This is my new DCR'

Data collection rules

Action	Command
Get rules	Get-AzDataCollectionRule
Create a rule	New-AzDataCollectionRule
Update a rule	Update-AzDataCollectionRule
Delete a rule	Remove-AzDataCollectionRule
Update "Tags" for a rule	Update-AzDataCollectionRule

Data collection rule associations

Action	Command
Get associations	Get-AzDataCollectionRuleAssociation
Create an association	New-AzDataCollectionRuleAssociation
Delete an association	Remove-AzDataCollectionRuleAssociation

API

Use the DCR create API to create the DCR from your JSON file. You can use any method to call a REST API as shown in the following examples.

$ResourceId = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.Insights/dataCollectionRules/my-dcr"
$FilePath = ".\my-dcr.json"
$DCRContent = Get-Content $FilePath -Raw 
Invoke-AzRestMethod -Path ("$ResourceId"+"?api-version=2022-06-01") -Method PUT -Payload $DCRContent

ResourceId="/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.Insights/dataCollectionRules/my-dcr"
FilePath="my-dcr.json"
az rest --method put --url $ResourceId"?api-version=2022-06-01" --body @$FilePath

ARM

See the following references for defining DCRs and associations in a template.

• Data collection rules
• Data collection rule associations

Use the following template to create a DCR using information from Structure of a data collection rule in Azure Monitor and Sample data collection rules (DCRs) in Azure Monitor to define the dcr-properties.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "dataCollectionRuleName": {
            "type": "string",
            "metadata": {
                "description": "Specifies the name of the Data Collection Rule to create."
            }
        },
        "location": {
            "type": "string",
            "metadata": {
                "description": "Specifies the location in which to create the Data Collection Rule."
            }
        }
    },
    "resources": [
        {
            "type": "Microsoft.Insights/dataCollectionRules",
            "name": "[parameters('dataCollectionRuleName')]",
            "location": "[parameters('location')]",
            "apiVersion": "2021-09-01-preview",
            "properties": {
                "<dcr-properties>"
            }
        }
    ]
}

DCR Association -Azure VM

The following sample creates an association between an Azure virtual machine and a data collection rule.

Bicep template file

@description('The name of the virtual machine.')
param vmName string

@description('The name of the association.')
param associationName string

@description('The resource ID of the data collection rule.')
param dataCollectionRuleId string

resource vm 'Microsoft.Compute/virtualMachines@2021-11-01' existing = {
  name: vmName
}

resource association 'Microsoft.Insights/dataCollectionRuleAssociations@2021-09-01-preview' = {
  name: associationName
  scope: vm
  properties: {
    description: 'Association of data collection rule. Deleting this association will break the data collection for this virtual machine.'
    dataCollectionRuleId: dataCollectionRuleId
  }
}

ARM template file

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "vmName": {
      "type": "string",
      "metadata": {
        "description": "The name of the virtual machine."
      }
    },
    "associationName": {
      "type": "string",
      "metadata": {
        "description": "The name of the association."
      }
    },
    "dataCollectionRuleId": {
      "type": "string",
      "metadata": {
        "description": "The resource ID of the data collection rule."
      }
    }
  },
  "resources": [
    {
      "type": "Microsoft.Insights/dataCollectionRuleAssociations",
      "apiVersion": "2021-09-01-preview",
      "scope": "[format('Microsoft.Compute/virtualMachines/{0}', parameters('vmName'))]",
      "name": "[parameters('associationName')]",
      "properties": {
        "description": "Association of data collection rule. Deleting this association will break the data collection for this virtual machine.",
        "dataCollectionRuleId": "[parameters('dataCollectionRuleId')]"
      }
    }
  ]
}

Parameter file

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "vmName": {
      "value": "my-azure-vm"
    },
    "associationName": {
      "value": "my-windows-vm-my-dcr"
    },
    "dataCollectionRuleId": {
      "value": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/my-resource-group/providers/microsoft.insights/datacollectionrules/my-dcr"
    }
   }
}

DCR Association -Arc-enabled server

The following sample creates an association between an Azure Arc-enabled server and a data collection rule.

Bicep template file

@description('The name of the virtual machine.')
param vmName string

@description('The name of the association.')
param associationName string

@description('The resource ID of the data collection rule.')
param dataCollectionRuleId string

resource vm 'Microsoft.HybridCompute/machines@2021-11-01' existing = {
  name: vmName
}

resource association 'Microsoft.Insights/dataCollectionRuleAssociations@2021-09-01-preview' = {
  name: associationName
  scope: vm
  properties: {
    description: 'Association of data collection rule. Deleting this association will break the data collection for this Arc server.'
    dataCollectionRuleId: dataCollectionRuleId
  }
}

ARM template file

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "vmName": {
      "type": "string",
      "metadata": {
        "description": "The name of the virtual machine."
      }
    },
    "associationName": {
      "type": "string",
      "metadata": {
        "description": "The name of the association."
      }
    },
    "dataCollectionRuleId": {
      "type": "string",
      "metadata": {
        "description": "The resource ID of the data collection rule."
      }
    }
  },
  "resources": [
    {
      "type": "Microsoft.Insights/dataCollectionRuleAssociations",
      "apiVersion": "2021-09-01-preview",
      "scope": "[format('Microsoft.HybridCompute/machines/{0}', parameters('vmName'))]",
      "name": "[parameters('associationName')]",
      "properties": {
        "description": "Association of data collection rule. Deleting this association will break the data collection for this Arc server.",
        "dataCollectionRuleId": "[parameters('dataCollectionRuleId')]"
      }
    }
  ]
}

Parameter file

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "vmName": {
      "value": "my-hybrid-vm"
    },
    "associationName": {
      "value": "my-windows-vm-my-dcr"
    },
    "dataCollectionRuleId": {
      "value": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/my-resource-group/providers/microsoft.insights/datacollectionrules/my-dcr"
    }
   }
}

Edit a DCR

To edit a DCR, you can use any of the methods described in the previous section to create a DCR using a modified version of the JSON.

If you need to retrieve the JSON for an existing DCR, you can copy it from the JSON View for the DCR in the Azure portal. You can also retrieve it using an API call as shown in the following PowerShell example.

$ResourceId = "<ResourceId>" # Resource ID of the DCR to edit
$FilePath = "<FilePath>" # Store DCR content in this file
$DCR = Invoke-AzRestMethod -Path ("$ResourceId"+"?api-version=2022-06-01") -Method GET
$DCR.Content | ConvertFrom-Json | ConvertTo-Json -Depth 20 | Out-File -FilePath $FilePath

For a tutorial that walks through the process of retrieving and then editing an existing DCR, see Tutorial: Edit a data collection rule (DCR).

Next steps

• Read about the detailed structure of a data collection rule
• Get details on transformations in a data collection rule