Data Collection Rules - Create

Creates or updates a data collection rule.

PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Insights/dataCollectionRules/{dataCollectionRuleName}?api-version=2021-09-01-preview

URI Parameters

Name In Required Type Description
dataCollectionRuleName
path True
  • string

The name of the data collection rule. The name is case insensitive.

resourceGroupName
path True
  • string

The name of the resource group. The name is case insensitive.

subscriptionId
path True
  • string

The ID of the target subscription.

api-version
query True
  • string

The API version to use for this operation.

Request Body

Name Required Type Description
location True
  • string

The geo-location where the resource lives.

kind

The kind of the resource.

properties.dataCollectionEndpointId
  • string

The resource ID of the data collection endpoint that this rule can be used with.

properties.dataFlows

The specification of data flows.

properties.dataSources

The specification of data sources. This property is optional and can be omitted if the rule is meant to be used via direct calls to the provisioned endpoint.

properties.description
  • string

Description of the data collection rule.

properties.destinations

The specification of destinations.

properties.streamDeclarations

Declaration of custom streams used in this rule.

tags
  • object

Resource tags.

Responses

Name Type Description
200 OK

Data collection rule was successfully updated

201 Created

Data collection rule was successfully created

Other Status Codes

Error

Security

azure_auth

Azure Active Directory OAuth2 Flow

Type: oauth2
Flow: implicit
Authorization URL: https://login.microsoftonline.com/common/oauth2/authorize

Scopes

Name Description
user_impersonation impersonate your user account

Examples

Create or update data collection rule

Sample Request

PUT https://management.azure.com/subscriptions/703362b3-f278-4e4b-9179-c76eaf41ffc2/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionRules/myCollectionRule?api-version=2021-09-01-preview

{
  "location": "eastus",
  "properties": {
    "dataSources": {
      "performanceCounters": [
        {
          "name": "cloudTeamCoreCounters",
          "streams": [
            "Microsoft-Perf"
          ],
          "samplingFrequencyInSeconds": 15,
          "counterSpecifiers": [
            "\\Processor(_Total)\\% Processor Time",
            "\\Memory\\Committed Bytes",
            "\\LogicalDisk(_Total)\\Free Megabytes",
            "\\PhysicalDisk(_Total)\\Avg. Disk Queue Length"
          ]
        },
        {
          "name": "appTeamExtraCounters",
          "streams": [
            "Microsoft-Perf"
          ],
          "samplingFrequencyInSeconds": 30,
          "counterSpecifiers": [
            "\\Process(_Total)\\Thread Count"
          ]
        }
      ],
      "windowsEventLogs": [
        {
          "name": "cloudSecurityTeamEvents",
          "streams": [
            "Microsoft-WindowsEvent"
          ],
          "xPathQueries": [
            "Security!"
          ]
        },
        {
          "name": "appTeam1AppEvents",
          "streams": [
            "Microsoft-WindowsEvent"
          ],
          "xPathQueries": [
            "System![System[(Level = 1 or Level = 2 or Level = 3)]]",
            "Application!*[System[(Level = 1 or Level = 2 or Level = 3)]]"
          ]
        }
      ],
      "syslog": [
        {
          "name": "cronSyslog",
          "streams": [
            "Microsoft-Syslog"
          ],
          "facilityNames": [
            "cron"
          ],
          "logLevels": [
            "Debug",
            "Critical",
            "Emergency"
          ]
        },
        {
          "name": "syslogBase",
          "streams": [
            "Microsoft-Syslog"
          ],
          "facilityNames": [
            "syslog"
          ],
          "logLevels": [
            "Alert",
            "Critical",
            "Emergency"
          ]
        }
      ]
    },
    "destinations": {
      "logAnalytics": [
        {
          "workspaceResourceId": "/subscriptions/703362b3-f278-4e4b-9179-c76eaf41ffc2/resourceGroups/myResourceGroup/providers/Microsoft.OperationalInsights/workspaces/centralTeamWorkspace",
          "name": "centralWorkspace"
        }
      ]
    },
    "dataFlows": [
      {
        "streams": [
          "Microsoft-Perf",
          "Microsoft-Syslog",
          "Microsoft-WindowsEvent"
        ],
        "destinations": [
          "centralWorkspace"
        ]
      }
    ]
  }
}

Sample Response

{
  "id": "/subscriptions/703362b3-f278-4e4b-9179-c76eaf41ffc2/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionRules/myCollectionRule",
  "name": "myCollectionRule",
  "type": "Microsoft.Insights/dataCollectionRules",
  "location": "eastus",
  "tags": {
    "tag1": "A",
    "tag2": "B"
  },
  "properties": {
    "immutableId": "dcr-b74e0d383fc9415abaa584ec41adece3",
    "dataSources": {
      "performanceCounters": [
        {
          "name": "cloudTeamCoreCounters",
          "streams": [
            "Microsoft-Perf"
          ],
          "samplingFrequencyInSeconds": 15,
          "counterSpecifiers": [
            "\\Processor(_Total)\\% Processor Time",
            "\\Memory\\Committed Bytes",
            "\\LogicalDisk(_Total)\\Free Megabytes",
            "\\PhysicalDisk(_Total)\\Avg. Disk Queue Length"
          ]
        },
        {
          "name": "appTeamExtraCounters",
          "streams": [
            "Microsoft-Perf"
          ],
          "samplingFrequencyInSeconds": 30,
          "counterSpecifiers": [
            "\\Process(_Total)\\Thread Count"
          ]
        }
      ],
      "windowsEventLogs": [
        {
          "name": "cloudSecurityTeamEvents",
          "streams": [
            "Microsoft-WindowsEvent"
          ],
          "xPathQueries": [
            "Security!"
          ]
        },
        {
          "name": "appTeam1AppEvents",
          "streams": [
            "Microsoft-WindowsEvent"
          ],
          "xPathQueries": [
            "System![System[(Level = 1 or Level = 2 or Level = 3)]]",
            "Application!*[System[(Level = 1 or Level = 2 or Level = 3)]]"
          ]
        }
      ],
      "syslog": [
        {
          "name": "cronSyslog",
          "streams": [
            "Microsoft-Syslog"
          ],
          "facilityNames": [
            "cron"
          ],
          "logLevels": [
            "Debug",
            "Critical",
            "Emergency"
          ]
        },
        {
          "name": "syslogBase",
          "streams": [
            "Microsoft-Syslog"
          ],
          "facilityNames": [
            "syslog"
          ],
          "logLevels": [
            "Alert",
            "Critical",
            "Emergency"
          ]
        }
      ]
    },
    "destinations": {
      "logAnalytics": [
        {
          "workspaceResourceId": "/subscriptions/703362b3-f278-4e4b-9179-c76eaf41ffc2/resourceGroups/myResourceGroup/providers/Microsoft.OperationalInsights/workspaces/centralTeamWorkspace",
          "workspaceId": "9ba8bc53-bd36-4156-8667-e983e7ae0e4f",
          "name": "centralWorkspace"
        }
      ]
    },
    "dataFlows": [
      {
        "streams": [
          "Microsoft-Perf",
          "Microsoft-Syslog",
          "Microsoft-WindowsEvent"
        ],
        "destinations": [
          "centralWorkspace"
        ]
      }
    ]
  },
  "systemData": {
    "createdBy": "user1",
    "createdByType": "User",
    "createdAt": "2021-04-01T12:34:56.1234567Z",
    "lastModifiedBy": "user2",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2021-04-02T12:34:56.1234567Z"
  },
  "etag": "070057da-0000-0000-0000-5ba70d6c0000"
}
{
  "id": "/subscriptions/703362b3-f278-4e4b-9179-c76eaf41ffc2/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionRules/myCollectionRule",
  "name": "myCollectionRule",
  "type": "Microsoft.Insights/dataCollectionRules",
  "location": "eastus",
  "tags": {
    "tag1": "A",
    "tag2": "B"
  },
  "properties": {
    "immutableId": "dcr-b74e0d383fc9415abaa584ec41adece3",
    "dataSources": {
      "performanceCounters": [
        {
          "name": "cloudTeamCoreCounters",
          "streams": [
            "Microsoft-Perf"
          ],
          "samplingFrequencyInSeconds": 15,
          "counterSpecifiers": [
            "\\Processor(_Total)\\% Processor Time",
            "\\Memory\\Committed Bytes",
            "\\LogicalDisk(_Total)\\Free Megabytes",
            "\\PhysicalDisk(_Total)\\Avg. Disk Queue Length"
          ]
        },
        {
          "name": "appTeamExtraCounters",
          "streams": [
            "Microsoft-Perf"
          ],
          "samplingFrequencyInSeconds": 30,
          "counterSpecifiers": [
            "\\Process(_Total)\\Thread Count"
          ]
        }
      ],
      "windowsEventLogs": [
        {
          "name": "cloudSecurityTeamEvents",
          "streams": [
            "Microsoft-WindowsEvent"
          ],
          "xPathQueries": [
            "Security!"
          ]
        },
        {
          "name": "appTeam1AppEvents",
          "streams": [
            "Microsoft-WindowsEvent"
          ],
          "xPathQueries": [
            "System![System[(Level = 1 or Level = 2 or Level = 3)]]",
            "Application!*[System[(Level = 1 or Level = 2 or Level = 3)]]"
          ]
        }
      ],
      "syslog": [
        {
          "name": "cronSyslog",
          "streams": [
            "Microsoft-Syslog"
          ],
          "facilityNames": [
            "cron"
          ],
          "logLevels": [
            "Debug",
            "Critical",
            "Emergency"
          ]
        },
        {
          "name": "syslogBase",
          "streams": [
            "Microsoft-Syslog"
          ],
          "facilityNames": [
            "syslog"
          ],
          "logLevels": [
            "Alert",
            "Critical",
            "Emergency"
          ]
        }
      ]
    },
    "destinations": {
      "logAnalytics": [
        {
          "workspaceResourceId": "/subscriptions/703362b3-f278-4e4b-9179-c76eaf41ffc2/resourceGroups/myResourceGroup/providers/Microsoft.OperationalInsights/workspaces/centralTeamWorkspace",
          "workspaceId": "9ba8bc53-bd36-4156-8667-e983e7ae0e4f",
          "name": "centralWorkspace"
        }
      ]
    },
    "dataFlows": [
      {
        "streams": [
          "Microsoft-Perf",
          "Microsoft-Syslog",
          "Microsoft-WindowsEvent"
        ],
        "destinations": [
          "centralWorkspace"
        ]
      }
    ]
  },
  "systemData": {
    "createdBy": "user1",
    "createdByType": "User",
    "createdAt": "2021-04-01T12:34:56.1234567Z",
    "lastModifiedBy": "user2",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2021-04-02T12:34:56.1234567Z"
  },
  "etag": "070057da-0000-0000-0000-5ba70d6c0000"
}

Definitions

AzureMonitorMetrics

Azure Monitor Metrics destination.

ColumnDefinition

Definition of custom data column.

createdByType

The type of identity that created the resource.

DataCollectionRuleResource

Definition of ARM tracked top level resource.

DataFlow

Definition of which streams are sent to which destinations.

DataSources

The specification of data sources. This property is optional and can be omitted if the rule is meant to be used via direct calls to the provisioned endpoint.

Destinations

The specification of destinations.

ErrorAdditionalInfo

The resource management error additional info.

ErrorDetail

The error detail.

ErrorResponseCommonV2

Error response

ExtensionDataSource

Definition of which data will be collected from a separate VM extension that integrates with the Azure Monitor Agent. Collected from either Windows and Linux machines, depending on which extension is defined.

IisLogsDataSource

Enables IIS logs to be collected by this data collection rule.

KnownColumnDefinitionType

The type of the column data.

KnownDataCollectionRuleProvisioningState

The resource provisioning state.

KnownDataCollectionRuleResourceKind

The kind of the resource.

KnownLogFilesDataSourceFormat

The data format of the log files

KnownLogFileTextSettingsRecordStartTimestampFormat

One of the supported timestamp formats

LogAnalyticsDestination

Log Analytics destination.

LogFilesDataSource

Definition of which custom log files will be collected by this data collection rule

Metadata

Metadata about the resource

PerfCounterDataSource

Definition of which performance counters will be collected and how they will be collected by this data collection rule. Collected from both Windows and Linux machines where the counter is present.

Settings

The log files specific settings.

StreamDeclaration

Declaration of a custom stream.

SyslogDataSource

Definition of which syslog data will be collected and how it will be collected. Only collected from Linux machines.

SystemData

Metadata pertaining to creation and last modification of the resource.

Text

Text settings

WindowsEventLogDataSource

Definition of which Windows Event Log events will be collected and how they will be collected. Only collected from Windows machines.

AzureMonitorMetrics

Azure Monitor Metrics destination.

Name Type Description
name
  • string

A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.

ColumnDefinition

Definition of custom data column.

Name Type Description
name
  • string

The name of the column.

type

The type of the column data.

createdByType

The type of identity that created the resource.

Name Type Description
Application
  • string
Key
  • string
ManagedIdentity
  • string
User
  • string

DataCollectionRuleResource

Definition of ARM tracked top level resource.

Name Type Description
etag
  • string

Resource entity tag (ETag).

id
  • string

Fully qualified ID of the resource.

kind

The kind of the resource.

location
  • string

The geo-location where the resource lives.

name
  • string

The name of the resource.

properties.dataCollectionEndpointId
  • string

The resource ID of the data collection endpoint that this rule can be used with.

properties.dataFlows

The specification of data flows.

properties.dataSources

The specification of data sources. This property is optional and can be omitted if the rule is meant to be used via direct calls to the provisioned endpoint.

properties.description
  • string

Description of the data collection rule.

properties.destinations

The specification of destinations.

properties.immutableId
  • string

The immutable ID of this data collection rule. This property is READ-ONLY.

properties.metadata

Metadata about the resource

properties.provisioningState

The resource provisioning state.

properties.streamDeclarations

Declaration of custom streams used in this rule.

systemData

Metadata pertaining to creation and last modification of the resource.

tags
  • object

Resource tags.

type
  • string

The type of the resource.

DataFlow

Definition of which streams are sent to which destinations.

Name Type Description
destinations
  • string[]

List of destinations for this data flow.

outputStream
  • string

The output stream of the transform. Only required if the transform changes data to a different stream.

streams
  • string[]

List of streams for this data flow.

transformKql
  • string

The KQL query to transform stream data.

DataSources

The specification of data sources. This property is optional and can be omitted if the rule is meant to be used via direct calls to the provisioned endpoint.

Name Type Description
extensions

The list of Azure VM extension data source configurations.

iisLogs

The list of IIS logs source configurations.

logFiles

The list of Log files source configurations.

performanceCounters

The list of performance counter data source configurations.

syslog

The list of Syslog data source configurations.

windowsEventLogs

The list of Windows Event Log data source configurations.

Destinations

The specification of destinations.

Name Type Description
azureMonitorMetrics

Azure Monitor Metrics destination.

logAnalytics

List of Log Analytics destinations.

ErrorAdditionalInfo

The resource management error additional info.

Name Type Description
info
  • object

The additional info.

type
  • string

The additional info type.

ErrorDetail

The error detail.

Name Type Description
additionalInfo

The error additional info.

code
  • string

The error code.

details

The error details.

message
  • string

The error message.

target
  • string

The error target.

ErrorResponseCommonV2

Error response

Name Type Description
error

The error object.

ExtensionDataSource

Definition of which data will be collected from a separate VM extension that integrates with the Azure Monitor Agent. Collected from either Windows and Linux machines, depending on which extension is defined.

Name Type Description
extensionName
  • string

The name of the VM extension.

extensionSettings
  • object

The extension settings. The format is specific for particular extension.

inputDataSources
  • string[]

The list of data sources this extension needs data from.

name
  • string

A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.

streams
  • string[]

List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.

IisLogsDataSource

Enables IIS logs to be collected by this data collection rule.

Name Type Description
logDirectories
  • string[]

Absolute paths file location

name
  • string

A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.

streams
  • string[]

IIS streams

KnownColumnDefinitionType

The type of the column data.

Name Type Description
boolean
  • string
datetime
  • string
dynamic
  • string
int
  • string
long
  • string
real
  • string
string
  • string

KnownDataCollectionRuleProvisioningState

The resource provisioning state.

Name Type Description
Creating
  • string
Deleting
  • string
Failed
  • string
Succeeded
  • string
Updating
  • string

KnownDataCollectionRuleResourceKind

The kind of the resource.

Name Type Description
Linux
  • string
Windows
  • string

KnownLogFilesDataSourceFormat

The data format of the log files

Name Type Description
text
  • string

KnownLogFileTextSettingsRecordStartTimestampFormat

One of the supported timestamp formats

Name Type Description
ISO 8601
  • string
M/D/YYYY HH:MM:SS AM/PM
  • string
MMM d hh:mm:ss
  • string
Mon DD, YYYY HH:MM:SS
  • string
YYYY-MM-DD HH:MM:SS
  • string
dd/MMM/yyyy:HH:mm:ss zzz
  • string
ddMMyy HH:mm:ss
  • string
yyMMdd HH:mm:ss
  • string
yyyy-MM-ddTHH:mm:ssK
  • string

LogAnalyticsDestination

Log Analytics destination.

Name Type Description
name
  • string

A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.

workspaceId
  • string

The Customer ID of the Log Analytics workspace.

workspaceResourceId
  • string

The resource ID of the Log Analytics workspace.

LogFilesDataSource

Definition of which custom log files will be collected by this data collection rule

Name Type Description
filePatterns
  • string[]

File Patterns where the log files are located

format

The data format of the log files

name
  • string

A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.

settings

The log files specific settings.

streams
  • string[]

List of streams that this data source will be sent to. A stream indicates what schema will be used for this data source

Metadata

Metadata about the resource

Name Type Description
provisionedBy
  • string

Azure offering managing this resource on-behalf-of customer.

PerfCounterDataSource

Definition of which performance counters will be collected and how they will be collected by this data collection rule. Collected from both Windows and Linux machines where the counter is present.

Name Type Description
counterSpecifiers
  • string[]

A list of specifier names of the performance counters you want to collect. Use a wildcard (*) to collect a counter for all instances. To get a list of performance counters on Windows, run the command 'typeperf'.

name
  • string

A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.

samplingFrequencyInSeconds
  • integer

The number of seconds between consecutive counter measurements (samples).

streams
  • string[]

List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.

Settings

The log files specific settings.

Name Type Description
text

Text settings

StreamDeclaration

Declaration of a custom stream.

Name Type Description
columns

List of columns used by data in this stream.

SyslogDataSource

Definition of which syslog data will be collected and how it will be collected. Only collected from Linux machines.

Name Type Description
facilityNames
  • string[]

The list of facility names.

logLevels
  • string[]

The log levels to collect.

name
  • string

A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.

streams
  • string[]

List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.

SystemData

Metadata pertaining to creation and last modification of the resource.

Name Type Description
createdAt
  • string

The timestamp of resource creation (UTC).

createdBy
  • string

The identity that created the resource.

createdByType

The type of identity that created the resource.

lastModifiedAt
  • string

The timestamp of resource last modification (UTC)

lastModifiedBy
  • string

The identity that last modified the resource.

lastModifiedByType

The type of identity that last modified the resource.

Text

Text settings

Name Type Description
recordStartTimestampFormat

One of the supported timestamp formats

WindowsEventLogDataSource

Definition of which Windows Event Log events will be collected and how they will be collected. Only collected from Windows machines.

Name Type Description
name
  • string

A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.

streams
  • string[]

List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.

xPathQueries
  • string[]

A list of Windows Event Log queries in XPATH format.