Migrate from Data Collector API and custom fields-enabled tables to DCR-based custom log collection

This article describes how to migrate from Data Collector API or custom fields in Azure Monitor to DCR-based custom log collection. It includes configuration required for custom tables created in your Log Analytics workspace so that they can be used by Logs ingestion API and workspace transformations.

Important

You do not need to follow this article if you are configuring your DCR-based custom logs using the Azure Portal since the configuration will be performed for you. This article only applies if you're configuring using Resource Manager templates APIs.

Background

To use a table with the Logs ingestion API or with a workspace transformation, it must be configured to support new features. When you complete the process described in this article, the following actions are taken:

  • The table is reconfigured to enable all DCR-based custom logs features. This includes DCR and DCE support and management with the new Tables control plane.
  • Any previously defined custom fields will stop populating.
  • The Data Collector API will continue to work but won't create any new columns. Data will only populate into any columns that was created prior to migration.
  • The schema and historic data is preserved and can be accessed the same way it was previously.

Applicable scenarios

This article is only applicable if all of the following criteria apply:

If all of these conditions aren't true, then you can use DCR-based log collection without following the procedure described here.

Migration procedure

If the table that you're targeting with DCR-based log collection fits the criteria above, then you must perform the following steps:

  1. Configure your data collection rule (DCR) following procedures at Send custom logs to Azure Monitor Logs using Resource Manager templates or Add transformation in workspace data collection rule to Azure Monitor using resource manager templates.

  2. If using the Logs ingestion API, also configure the data collection endpoint (DCE) and the agent or component that will be sending data to the API.

  3. Issue the following API call against your table. This call is idempotent, so there will be no effect if the table has already been migrated.

    POST /subscriptions/{subscriptionId}/resourcegroups/{resourceGroupName}/providers/microsoft.operationalinsights/workspaces/{workspaceName}/tables/{tableName}/migrate?api-version=2021-03-01-privatepreview
    
  4. Discontinue use of the Data Collector API and start using the new Logs ingestion API.

Next steps