Create a data collection rule (DCR) for metrics export
Article
This article describes how to create a data collection rule (DCR) for metrics export using the Azure portal, Azure CLI, PowerShell, API, or ARM templates.
Important
To send Platform Telemetry data to Storage Accounts or Event Hubs, the resource, data collection rule, and the destination Storage Account or the Event Hubs must all be in the same region.
On the Monitor menu in the Azure portal, select Data Collection Rules then select Create.
To create a DCR to collect platform metrics data, select the link on the top of the page.
On the Create Data Collection Rule page, enter a rule name, select a Subscription, Resource group, and Region for the DCR.
Select Enable Managed Identity if you want to send metrics to a Storage Account or Event Hubs.
Select Next
On the Resources page, select Add resources to add the resources you want to collect metrics from.
Select Next to move to the Collect and deliver tab.
Select Add new dataflow
The resource type of the resource that chose in the previous step is automatically selected. Add more resource types if you want to use this rule to collect metrics from multiple resource types in the future.
Select Next Destinations to move to the Destinations tab.
To send metrics to a Log Analytics workspace, select Azure Monitor Logs from the Destination type dropdown.
Select the Subscription and the Log Analytics workspace you want to send the metrics to.
To send metrics to Event Hubs, select Event Hub from the Destination type dropdown.
Select the Subscription, the Event Hub namespace, and the Event Hub instance name.
To send metrics to a Storage Account, select Storage Account from the Destination type dropdown.
Select the Subscription, the Storage Account, and the Blob container where you want to store the metrics.
Note
To sent metrics to a Storage Account or Event Hubs, the resource generating the metrics, the DCR, and the Storage Account or Event Hub, must all be in the same region.
To send metrics to a Log Analytics workspace, the DCR must be in the same region as the Log Analytics workspace. The resource generating the metrics can be in any region.
To select Storage Account or Event Hubs as the destination, you must enable managed identity for the DCR on the Basics tab.
The rule file has the same format as used for PowerShell and the REST API, however the file must not contain identity, the location, or kind. These parameters are specified in the az monitor data-collection rule create command.
Use the following command to create a data collection rule for metrics using the Azure CLI.
Azure CLI
az monitor data-collection rule create --name--resource-group--location--kind PlatformTelemetry
--rule-file
[--identity"{type:'SystemAssigned'}"]
For storage account and Event Hubs destinations, you must enable managed identity for the DCR using --identity "{type:'SystemAssigned'}". Identity isn't required for Log Analytics workspaces.
The managed identity used by the DCR must have write permissions to the destination when the destination is a Storage Account or Event Hubs.
To grant permissions for the rule's managed entity, assign the appropriate role to the entity.
The following table shows the roles required for each destination type:
Assign the appropriate role to the managed identity of the DCR.
Azure CLI
az role assignment create --assignee<system assigned principal ID> \
--role<`Storage Blob Data Contributor` or `Azure Event Hubs Data Sender` \
--scope <storage account ID or eventhub ID>
The following example assigns the Storage Blob Data Contributor role to the managed identity of the DCR for a storage account.
Azure CLI
az role assignment create --assignee eeeeeeee-ffff-aaaa-5555-666666666666 \
--role"Storage Blob Data Contributor" \
--scope /subscriptions/bbbb1b1b-cc2c-DD3D-ee4e-ffffff5f5f5f/resourceGroups/ed-rg-DCRTest/providers/Microsoft.Storage/storageAccounts/metricsexport001
Create a data collection rule association
After you create the data collection rule, create a data collection rule association (DCRA) to associate the rule with the resource to be monitored. For more information, see Data Collection Rule Associations - Create
Use az monitor data-collection rule association create to create an association between a data collection rule and a resource.
Azure CLI
az monitor data-collection rule association create --name--rule-id--resource
The following example creates an association between a data collection rule and a Key Vault.
Azure CLI
az monitor data-collection rule association create --name"keyValut-001" \
--rule-id"/subscriptions/bbbb1b1b-cc2c-DD3D-ee4e-ffffff5f5f5f/resourceGroups/rg-dcr/providers/Microsoft.Insights/dataCollectionRules/dcr-cli-001" \
--resource"/subscriptions/bbbb1b1b-cc2c-DD3D-ee4e-ffffff5f5f5f/resourceGroups/rg-dcr/providers/Microsoft.KeyVault/vaults/keyVault-001"
Use the New-AzDataCollectionRule command to create a data collection rule for metrics using PowerShell. For more information, see New-AzDataCollectionRule.
The managed identity used by the DCR must have write permissions to the destination when the destination is a Storage Account or Event Hubs.
To grant permissions for the rule's managed entity, assign the appropriate role to the entity.
The following table shows the roles required for each destination type:
The following example assigns the Azure Event Hubs Data Sender role to the managed identity of the DCR at the subscription level.
PowerShell
New-AzRoleAssignment -ObjectIdeeeeeeee-ffff-aaaa-5555-666666666666 -RoleDefinitionName"Azure Event Hubs Data Sender" -Scope /subscriptions/bbbb1b1b-cc2c-DD3D-ee4e-ffffff5f5f5f
Create a data collection rule association
After you create the data collection rule, create a data collection rule association (DCRA) to associate the rule with the resource to be monitored. Use New-AzDataCollectionRuleAssociation to create an association between a data collection rule and a resource. For more information, see New-AzDataCollectionRuleAssociation
Creating a data collection rule for metrics requires the following steps:
Create the data collection rule.
Grant permissions for the rule's managed entity to write to the destination
Create a data collection rule association.
Create the data collection rule
To create a DCR using the REST API, you must make an authenticated request using a bearer token. For more information on authenticating with Azure Monitor, see Authenticate Azure Monitor requests.
Use the following endpoint to create a data collection rule for metrics using the REST API.
For more information, see Data Collection Rules - Create.
HTTP
PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Insights/dataCollectionRules/{dataCollectionRuleName}?api-version=2023-03-11
The payload is a JSON object that defines a collection rule. The payload is sent in the body of the request. For more information on the JSON structure, see DCR specifications. For sample DCR JSON objects, see Sample Metrics Export JSON objects
Grant write permissions to the managed entity
The managed identity used by the DCR must have write permissions to the destination when the destination is a Storage Account or Event Hubs.
To grant permissions for the rule's managed entity, assign the appropriate role to the entity.
The following table shows the roles required for each destination type:
After you create the data collection rule, create a data collection rule association (DCRA) to associate the rule with the resource to be monitored. For more information, see Data Collection Rule Associations - Create
To create a DCRA using the REST API, use the following endpoint and payload:
HTTP
PUT https://management.azure.com/{resourceUri}/providers/Microsoft.Insights/dataCollectionRuleAssociations/{associationName}?api-version=2022-06-0
After creating the DCR and DCRA, allow up to 30 minutes for the first platform metrics data to appear in the Log Analytics Workspace. Once data starts flowing, the latency for a platform metric time series flowing to a Log Analytics workspace, Storage Account, or Event Hubs is approximately 3 minutes, depending on the resource type.
Verify and troubleshoot data collection
Once you install the DCR, it may take several minutes for the changes to take effect and data to be collected with the updated DCR. If you don't see any data being collected, it can be difficult to determine the root cause of the issue. Use the DCR monitoring features, which include metrics and logs to help troubleshoots.
DCR metrics are collected automatically for all DCRs, and you can analyze them using metrics explorer like the platform metrics for other Azure resources. Enable DCR error logs to get detailed error information when data processing is not successful.
If you don't see data being collected, follow these basic steps to troubleshoot the issue.
Check metrics such as Logs Ingestion Bytes per Min and Logs Rows Received per Min to ensure that the data is reaching Azure Monitor. If not, then check your data source to ensure that it's sending data as expected.
Check Logs Rows Dropped per Min to see if any rows are being dropped. This may not indicate an error since the rows could be dropped by a transformation. If the rows dropped is the same as Logs Rows Dropped per Min though, then no data will be ingested in the workspace. Examine the Logs Transformation Errors per Min to see if there are any transformation errors.
Check Logs Transformation Errors per Min to determine if there are any errors from transformations applied to the incoming data. This could be due to changes in the data structure or the transformation itself.
Check the DCRLogErrors table for any ingestion errors that may have been logged. This can provide additional detail in identifying the root cause of the issue.