Migrate from diagnostic settings storage retention to Azure Storage lifecycle management

The Diagnostic Settings Storage Retention feature is being deprecated. To configure retention for logs and metrics sent to an Azure Storage account, use Azure Storage Lifecycle Management.

This guide walks you through migrating from using Azure diagnostic settings storage retention to using Azure Storage lifecycle management for retention. For logs sent to a Log Analytics workspace, retention is set for each table on the Tables page of your workspace.

Important

Deprecation Timeline.

• March 31, 2023 – The Diagnostic Settings Storage Retention feature will no longer be available to configure new retention rules for log data. This includes using the portal, CLI PowerShell, and ARM and Bicep templates. If you have configured retention settings, you'll still be able to see and change them in the portal.
• March 31, 2024 – You will no longer be able to use the API (CLI, Powershell, or templates), or Azure portal to configure retention setting unless you're changing them to 0. Existing retention rules will still be respected.
• September 30, 2025 – All retention functionality for the Diagnostic Settings Storage Retention feature will be disabled across all environments.

Prerequisites

An existing diagnostic setting logging to a storage account.

Migration Procedures

Azure portal

To migrate your diagnostics settings retention rules, follow the steps below:

1. Go to the Diagnostic Settings page for your logging resource and locate the diagnostic setting you wish to migrate

2. Set the retention for your logged categories to 0

3. Select Save

4. Navigate to the storage account you're logging to

5. Under Data management, select Lifecycle Management to view or change lifecycle management policies

6. Select List View, and select Add a rule

7. Enter a Rule name

8. Under Rule Scope, select Limit blobs with filters

9. Under Blob Type, select Append Blobs and Base blobs under Blob subtype.

10. Select Next

11. Set your retention time, then select Next

12. On the Filters tab, under Blob prefix set path or prefix to the container or logs you want the retention rule to apply to. The path or prefix can be at any level within the container and will apply to all blobs under that path or prefix. For example, for all insight activity logs, use the container insights-activity-logs to set the retention for all of the log in that container logs.
    To set the rule for a specific webapp app, use insights-activity-logs/ResourceId=/SUBSCRIPTIONS/<your subscription Id>/RESOURCEGROUPS/<your resource group>/PROVIDERS/MICROSOFT.WEB/SITES/<your webapp name>.

    Use the Storage browser to help you find the path or prefix.
    The example below shows the prefix for a specific web app: *insights-activity-logs/ResourceId=/SUBSCRIPTIONS/d05145d-4a5d-4a5d-4a5d-5267eae1bbc7/RESOURCEGROUPS/rg-001/PROVIDERS/MICROSOFT.WEB/SITES/appfromdocker1.
    To set the rule for all resources in the resource group, use insights-activity-logs/ResourceId=/SUBSCRIPTIONS/d05145d-4a5d-4a5d-4a5d-5267eae1bbc7/RESOURCEGROUPS/rg-001.

13. Select Add to save the rule.

CLI

Use the az storage account management-policy create command to create a lifecycle management policy. You must still set the retention in your diagnostic settings to 0. See the Azure portal section above for more information.

az storage account management-policy create   --account-name <storage account name> --resource-group <resource group name> --policy @<policy definition file>

The sample policy definition file below sets the retention for all blobs in the container insights-activity-logs for the given subscription ID. For more information, see Lifecycle management policy definition.

{
  "rules": [
    {
      "enabled": true,
      "name": "Susbcription level lifecycle rule",
      "type": "Lifecycle",
      "definition": {
        "actions": {
          "baseBlob": {
              "delete": {
              "daysAfterModificationGreaterThan": 120
            }
          }
        },
        "filters": {
          "blobTypes": [
            "appendBlob"
          ],
          "prefixMatch": [
            "insights-activity-logs/ResourceId=/SUBSCRIPTIONS/ABCD1234-5849-ABCD-1234-9876543210AB"
          ]
        }
      }
    }
  ]
}

Templates

Apply the following template to create a lifecycle management policy. You must still set the retention in your diagnostic settings to 0. See the Azure portal section above for more information.

az deployment group create  --resource-group <resource group name> --template-file <template file>

The following template sets the retention for storage account azmonstorageaccount001 for all blobs in the container insights-activity-logs for all resources for the subscription ID ABCD1234-5849-ABCD-1234-9876543210AB.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "resources": [
        {
            "type": "Microsoft.Storage/storageAccounts/managementPolicies",
            "apiVersion": "2021-02-01",
            "name": "azmonstorageaccount001/default",
            "properties": {
                "policy": {
                    "rules": [
                        {
                            "enabled": true,
                            "name": "Susbcription level lifecycle rule",
                            "type": "Lifecycle",
                            "definition": {
                                "actions": {
                                    "baseBlob": {
                                        "delete": {
                                            "daysAfterModificationGreaterThan": 120
                                        }
                                    }
                                },
                                "filters": {
                                    "blobTypes": [
                                        "appendBlob"
                                    ],
                                    "prefixMatch": [
                                        "insights-activity-logs/ResourceId=/SUBSCRIPTIONS/ABCD1234-5849-ABCD-1234-9876543210AB"
                                    ]
                                }
                            }
                        }
                    ]
                }
            }
        }
    ]
}

Next steps

Configure a lifecycle management policy.