Data retention and archive in Azure Monitor Logs

Azure Monitor Logs retains data in two states:

  • Interactive retention: Lets you retain Analytics logs for interactive queries of up to 2 years.
  • Archive: Lets you keep older, less used data in your workspace at a reduced cost. You can access data in the archived state by using search jobs and restore. You can keep data in archived state for up to 12 years.

This article describes how to configure data retention and archiving.

How retention and archiving work

Each workspace has a default retention setting that's applied to all tables. You can configure a different retention setting on individual tables.

Diagram that shows an overview of data retention and archive periods.

During the interactive retention period, data is available for monitoring, troubleshooting, and analytics. When you no longer use the logs, but still need to keep the data for compliance or occasional investigation, archive the logs to save costs.

Archived data stays in the same table, alongside the data that's available for interactive queries. When you set a total retention period that's longer than the interactive retention period, Log Analytics automatically archives the relevant data immediately at the end of the retention period.

You can access archived data by running a search job or restoring archived logs.

Note

The archive period can only be set at the table level, not at the workspace level.

Adjustments to retention and archive settings

When you shorten an existing retention setting, Azure Monitor waits 30 days before removing the data, so you can revert the change and avoid data loss in the event of an error in configuration. You can purge data immediately when required.

When you increase the retention setting, the new retention period applies to all data that's already been ingested into the table and hasn't yet been purged or removed.

If you change the archive settings on a table with existing data, the relevant data in the table is also affected immediately. For example, you might have an existing table with 180 days of interactive retention and no archive period. You decide to change the retention setting to 90 days of interactive retention without changing the total retention period of 180 days. Log Analytics immediately archives any data that's older than 90 days and none of the data is deleted.

Permissions required

Action Permissions required
Configure data retention and archive policies for a Log Analytics workspace Microsoft.OperationalInsights/workspaces/write and microsoft.operationalinsights/workspaces/tables/write permissions to the Log Analytics workspace, as provided by the Log Analytics Contributor built-in role, for example
Get the retention and archive policy by table for a Log Analytics workspace Microsoft.OperationalInsights/workspaces/tables/read permissions to the Log Analytics workspace, as provided by the Log Analytics Reader built-in role, for example
Purge data from a Log Analytics workspace Microsoft.OperationalInsights/workspaces/purge/action permissions to the Log Analytics workspace, as provided by the Log Analytics Contributor built-in role, for example
Set data retention for a classic Application Insights resource microsoft.insights/components/write permissions to the classic Application Insights resource, as provided by the Application Insights Component Contributor built-in role, for example
Purge data from a classic Application Insights resource Microsoft.Insights/components/purge/action permissions to the classic Application Insights resource, as provided by the Application Insights Component Contributor built-in role, for example

Configure the default workspace retention

You can set a Log Analytics workspace's default retention in the Azure portal to 30, 31, 60, 90, 120, 180, 270, 365, 550, and 730 days. You can apply a different setting to specific tables by configuring retention and archive at the table level. If you're on the free tier, you need to upgrade to the paid tier to change the data retention period.

To set the default workspace retention:

  1. From the Log Analytics workspaces menu in the Azure portal, select your workspace.

  2. Select Usage and estimated costs in the left pane.

  3. Select Data Retention at the top of the page.

    Screenshot that shows changing the workspace data retention setting.

  4. Move the slider to increase or decrease the number of days, and then select OK.

Configure retention and archive at the table level

By default, all tables in your workspace inherit the workspace's interactive retention setting and have no archive. You can modify the retention and archive settings of individual tables, except for workspaces in the legacy Free Trial pricing tier.

The Analytics log data plan includes 30 days of interactive retention. You can increase the interactive retention period to up to 730 days at an additional cost. If needed, you can reduce the interactive retention period to as little as four days using the API or CLI. However, since 30 days are included in the ingestion price, lowering the retention period below 30 days doesn't reduce costs. You can set the archive period to a total retention time of up to 4,383 days (12 years).

Note

Currently, you can set total retention to up to 12 years through the Azure portal and API. CLI and PowerShell are limited to seven years; support for 12 years will follow.

To set the retention and archive duration for a table in the Azure portal:

  1. From the Log Analytics workspaces menu, select Tables.

    The Tables screen lists all the tables in the workspace.

  2. Select the context menu for the table you want to configure and select Manage table.

    Screenshot that shows the Manage table button for one of the tables in a workspace.

  3. Configure the retention and archive duration in the Data retention settings section of the table configuration screen.

    Screenshot that shows the data retention settings on the table configuration screen.

Get retention and archive settings by table

To view the retention and archive duration for a table in the Azure portal, from the Log Analytics workspaces menu, select Tables.

The Tables screen shows the interactive retention and archive period for all the tables in the workspace.

Screenshot that shows the Manage table button for one of the tables in a workspace.

Purge retained data

If you set the data retention to 30 days, you can purge older data immediately by using the immediatePurgeDataOn30Days parameter in Azure Resource Manager. The purge functionality is useful when you need to remove personal data immediately. The immediate purge functionality isn't available through the Azure portal.

Workspaces with a 30-day retention might keep data for 31 days if you don't set the immediatePurgeDataOn30Days parameter.

You can also purge data from a workspace by using the purge feature, which removes personal data. You can't purge data from archived logs.

Important

The Log Analytics Purge feature doesn't affect your retention costs. To lower retention costs, decrease the retention period for the workspace or for specific tables.

Tables with unique retention periods

By default, two data types, Usage and AzureActivity, keep data for at least 90 days at no charge. When you increase the workspace retention to more than 90 days, you also increase the retention of these data types. These tables are also free from data ingestion charges.

Tables related to Application Insights resources also keep data for 90 days at no charge. You can adjust the retention of each of these tables individually:

  • AppAvailabilityResults
  • AppBrowserTimings
  • AppDependencies
  • AppExceptions
  • AppEvents
  • AppMetrics
  • AppPageViews
  • AppPerformanceCounters
  • AppRequests
  • AppSystemEvents
  • AppTraces

Pricing model

The charge for maintaining archived logs is calculated based on the volume of data you archive, in GB, and the number or days for which you archive the data. Log data that has _IsBillable == false is not subject to retention or archive charges.

For more information, see Azure Monitor pricing.

Set data retention for classic Application Insights resources

Workspace-based Application Insights resources store data in a Log Analytics workspace, so it's included in the data retention and archive settings for the workspace. Classic Application Insights resources have separate retention settings.

The default retention for Application Insights resources is 90 days. You can select different retention periods for each Application Insights resource. The full set of available retention periods is 30, 60, 90, 120, 180, 270, 365, 550, or 730 days.

To change the retention, from your Application Insights resource, go to the Usage and estimated costs page and select the Data retention option.

Screenshot that shows where to change the data retention period.

A several-day grace period begins when the retention is lowered before the oldest data is removed.

The retention can also be set programmatically with PowerShell by using the retentionInDays parameter. If you set the data retention to 30 days, you can trigger an immediate purge of older data by using the immediatePurgeDataOn30Days parameter. This approach might be useful for compliance-related scenarios. This purge functionality is only exposed via Azure Resource Manager and should be used with extreme care. The daily reset time for the data volume cap can be configured by using Azure Resource Manager to set the dailyQuotaResetTime parameter.

Next steps