Set up a table with the Auxiliary plan in your Log Analytics workspace (Preview)
The Auxiliary table plan lets you ingest and retain data in your Log Analytics workspace at a low cost. Azure Monitor Logs currently supports the Auxiliary table plan on data collection rule (DCR)-based custom tables to which you send data you collect using Azure Monitor Agent or the Logs ingestion API.
This article explains how to create a custom table with the Auxiliary plan in your Log Analytics workspace and set up a data collection rule that sends data to this table.
Here's a video that explains some of the uses and benefits of the Auxiliary table plan:
Important
See public preview limitations for supported regions and limitations related to Auxiliary tables and data collection rules.
Prerequisites
To create a custom table and collect log data, you need:
- A Log Analytics workspace where you have at least contributor rights.
- A data collection endpoint (DCE).
- All tables in a Log Analytics workspace have a column named
TimeGenerated. If your raw log data has aTimeGeneratedproperty, Azure Monitor uses this value to identify the creation time of the record. For a table with the Auxiliary plan, theTimeGeneratedcolumn currently supports ISO8601 format only. For information about theTimeGeneratedformat, see supported ISO 8601 datetime format.
Create a custom table with the Auxiliary plan
To create a custom table, call the Tables - Create Or Update API by using this command:
https://management.azure.com/subscriptions/{subscription_id}/resourceGroups/{resource_group}/providers/Microsoft.OperationalInsights/workspaces/{workspace_name}/tables/{table name_CL}?api-version=2023-01-01-preview
Important
Only version 2023-01-01-preview of the API currently lets you set the Auxiliary table plan.
Provide this payload - update the table name and adjust the columns based on your table schema:
{
"properties": {
"schema": {
"name": "table_name_CL",
"columns": [
{
"name": "TimeGenerated",
"type": "datetime"
},
{
"name": "StringProperty",
"type": "string"
},
{
"name": "IntProperty",
"type": "int"
},
{
"name": "LongProperty",
"type": "long"
},
{
"name": "RealProperty",
"type": "real"
},
{
"name": "BooleanProperty",
"type": "boolean"
},
{
"name": "GuidProperty",
"type": "real"
},
{
"name": "DateTimeProperty",
"type": "datetime"
}
]
},
"totalRetentionInDays": 365,
"plan": "Auxiliary"
}
}
Send data to a table with the Auxiliary plan
There are currently two ways to ingest data to a custom table with the Auxiliary plan:
Collect logs from a text file with Azure Monitor Agent / Collect logs from a JSON file with Azure Monitor Agent.
If you use this method, your custom table must only have two columns -
TimeGeneratedandRawData(of typestring). The data collection rule sends the entirety of each log entry you collect to theRawDatacolumn, and Azure Monitor Logs automatically populates theTimeGeneratedcolumn with the time the log is ingested.Send data to Azure Monitor using Logs ingestion API.
To use this method:
Create a custom table with the Auxiliary plan as described in this article.
Follow the steps described in Tutorial: Send data to Azure Monitor using Logs ingestion API to:
- Create a Microsoft Entra application.
- Create a data collection rule using this ARM template.
{ "$schema": "https://schema.management.azure.com/schemas/2019-08-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "dataCollectionRuleName": { "type": "string", "metadata": { "description": "Specifies the name of the data collection rule to create." } }, "location": { "type": "string", "metadata": { "description": "Specifies the region in which to create the data collection rule. The must be the same region as the destination Log Analytics workspace." } }, "workspaceResourceId": { "type": "string", "metadata": { "description": "The Azure resource ID of the Log Analytics workspace in which you created a custom table with the Auxiliary plan." } } }, "resources": [ { "type": "Microsoft.Insights/dataCollectionRules", "name": "[parameters('dataCollectionRuleName')]", "location": "[parameters('location')]", "apiVersion": "2023-03-11", "kind": "Direct", "properties": { "streamDeclarations": { "Custom-table_name_CL": { "columns": [ { "name": "TimeGenerated", "type": "datetime" }, { "name": "StringProperty", "type": "string" }, { "name": "IntProperty", "type": "int" }, { "name": "LongProperty", "type": "long" }, { "name": "RealProperty", "type": "real" }, { "name": "BooleanProperty", "type": "boolean" }, { "name": "GuidProperty", "type": "real" }, { "name": "DateTimeProperty", "type": "datetime" } ] } }, "destinations": { "logAnalytics": [ { "workspaceResourceId": "[parameters('workspaceResourceId')]", "name": "myworkspace" } ] }, "dataFlows": [ { "streams": [ "Custom-table_name_CL" ], "destinations": [ "myworkspace" ] } ] } } ], "outputs": { "dataCollectionRuleId": { "type": "string", "value": "[resourceId('Microsoft.Insights/dataCollectionRules', parameters('dataCollectionRuleName'))]" } } }Where:
myworkspaceis the name of your Log Analytics workspace.table_name_CLis the name of your table.columnsincludes the same columns you set in Create a custom table with the Auxiliary plan.
Public preview limitations
During public preview, these limitations apply:
The Auxiliary plan is gradually being rolled out to all regions and is currently supported in:
Region Locations Americas Canada Central Central US East US East US 2 West US West US 2 South Central US North Central US Asia Pacific Australia East Australia South East East Asia Europe West Europe North Europe UK South Germany West Central Switzerland North France Central Middle East Israel Central You can set the Auxiliary plan only on data collection rule-based custom tables you create using the Tables - Create Or Update API, version
2023-01-01-preview.Tables with the Auxiliary plan:
- Are currently unbilled. There's currently no charge for ingestion, queries, search jobs, and long-term retention.
- Do not support columns with dynamic data.
- Have a fixed total retention of 365 days.
- Support ISO 8601 datetime format only.
A data collection rule that sends data to a table with an Auxiliary plan:
- Can only send data to a single table.
- Can't include a transformation.
Ingestion data for Auxiliary tables isn't currently available in the Azure Monitor Logs Usage table. To estimate data ingestion volume, you can count the number of records in your Auxiliary table using this query:
MyTable_CL | summarize count()These features are currently not supported:
Feature Details Log Analytics workspace replication Azure Monitor doesn't replicate data in tables with the Auxiliary plan to your secondary workspace. Therefore, this data isn't protected against data loss in the event of a regional failure and isn't available when you swith over to your secondary workspace. Customer-managed keys Data in tables with the Auxiliary plan is encrypted with Microsoft-managed keys, even if you protect the data in the rest of your Log Analytics workspace using your own encryption key. Customer Lockbox for Microsoft Azure The Lockbox interface, which lets you review and approve or reject customer data access requests in response to a customer-initiated support ticket or a problem identified by Microsoft does not apply to tables with the Auxiliary plan.
Next steps
Learn more about: