Edit

Monitor Azure Cache for Redis data using diagnostic settings

  • Article

Diagnostic settings in Azure are used to collect resource logs. An Azure resource emits resource logs and provides rich, frequent data about the operation of that resource. These logs are captured per request and are also referred to as "data plane logs". See diagnostic settings in Azure Monitor for a recommended overview of the functionality in Azure. The content of these logs varies by resource type. In Azure Cache for Redis, two options are available to log:

  • Cache Metrics (that is "AllMetrics") used to log metrics from Azure Monitor
  • Connection Logs logs connections to the cache for security and diagnostic purposes.

Scope of availability

Tier Basic, Standard, and Premium Enterprise and Enterprise Flash
Cache Metrics Yes Yes
Connection Logs Yes Yes

Cache Metrics

Azure Cache for Redis emits many metrics such as Server Load and Connections per Second that are useful to log. Selecting the AllMetrics option allows these and other cache metrics to be logged. You can configure how long the metrics are retained. See here for an example of exporting cache metrics to a storage account.

Connection Logs

Azure Cache for Redis uses Azure diagnostic settings to log information on client connections to your cache. Logging and analyzing this diagnostic setting helps you understand who is connecting to your caches and the timestamp of those connections. The log data could be used to identify the scope of a security breach and for security auditing purposes.

Differences Between Azure Cache for Redis Tiers

Implementation of connection logs is slightly different between tiers:

  • Basic, Standard, and Premium-tier caches polls client connections by IP address, including the number of connections originating from each unique IP address. These logs aren't cumulative. They represent point-in-time snapshots taken at 10-second intervals. Authentication events (successful and failed) and disconnection events aren't logged in these tiers.
  • Enterprise and Enterprise Flash-tier caches use the audit connection events functionality built-into Redis Enterprise. Audit connection events allow every connection, disconnection, and authentication event to be logged, including failed authentication events.

The connection logs produced look similar among the tiers, but have some differences. The two formats are shown in more detail later in the article.

Important

The connection logging in the Basic, Standard, and Premium tiers polls the current client connections in the cache. The same client IP addresses appears over and over again. Logging in the Enterprise and Enterprise Flash tiers is focused on each connection event. Logs only occur when the actual event occurred for the first time.

Prerequisites/Limitations of Connection Logging

Basic, Standard, and Premium tiers

  • Because connection logs in these tiers consist of point-in-time snapshots taken every 10 seconds, connections that are established and removed in-between 10-second intervals aren't logged.
  • Authentication events aren't logged.
  • All diagnostic settings may take up to 90 minutes to start flowing to your selected destination.
  • Enabling connection logs can cause a small performance degradation to the cache instance.
  • Only the Analytics Logs pricing plan is supported when streaming logs to Azure Log Analytics. For more information, see Azure Monitor pricing.

Enterprise and Enterprise Flash tiers

  • When you use OSS Cluster Policy, logs are emitted from each data node. When you use Enterprise Cluster Policy, only the node being used as a proxy emits logs. Both versions still cover all connections to the cache. This is just an architectural difference.
  • Data loss (that is, missing a connection event) is rare, but possible. Data loss is typically caused by networking issues.
  • Disconnection logs aren't yet fully stable and events may be missed.
  • Because connection logs on the Enterprise tiers are event-based, be careful of your retention policies. For instance, if retention is set to 10 days, and a connection event occurred 15 days ago, that connection might still exist, but the log for that connection isn't retained.
  • If using active geo-replication, logging must be configured for each cache instance in the geo-replication group individually.
  • All diagnostic settings may take up to 90 minutes to start flowing to your selected destination.
  • Enabling connection logs may cause a small performance degradation to the cache instance.

Note

It is always possible to use the INFO or CLIENT LIST commands to check who is connected to a cache instance on-demand.

Important

When selecting logs, you can chose either the specific Category or Category groups, which are predefined groupings of logs across Azure services. When you use Category groups, you can no longer configure the retention settings. If you need to determine retention duration for your connection logs, select the item in the Categories section instead.

Log Destinations

You can turn on diagnostic settings for Azure Cache for Redis instances and send resource logs to the following destinations:

  • Log Analytics workspace - doesn't need to be in the same region as the resource being monitored.
  • Storage account - must be in the same region as the cache. Premium storage accounts are not supported as a destination, however.
  • Event hub - diagnostic settings can't access event hub resources when virtual networks are enabled. Enable the Allow trusted Microsoft services to bypass this firewall? setting in event hubs to grant access to your event hub resources. The event hub must be in the same region as the cache.
  • Partner Solution - a list of potential partner logging solutions can be found here

For more information on diagnostic requirements, see diagnostic settings.

You're charged normal data rates for storage account and event hub usage when you send diagnostic logs to either destination. You're billed under Azure Monitor not Azure Cache for Redis. When sending logs to Log Analytics, you're only charged for Log Analytics data ingestion.

For more pricing information, Azure Monitor pricing.

Enable connection logging using the Azure portal

  1. Sign in to the Azure portal.

  2. Navigate to your Azure Cache for Redis account. Open the Diagnostic settings pane under the Monitoring section on the left. Then, select Add diagnostic setting.

    Select diagnostics

  3. In the Diagnostic settings pane, select ConnectedClientList from Categories.

    For more detail on the data logged, see below Contents of the Connection Logs.

  4. Once you select ConnectedClientList, send your logs to your preferred destination. Select the information in the working pane.

    Select enable resource-specific

Enable connection logging using the REST API

Use the Azure Monitor REST API for creating a diagnostic setting via the interactive console. For more information, see Create or update.

Request

PUT https://management.azure.com/{resourceUri}/providers/Microsoft.Insights/diagnosticSettings/{name}?api-version=2017-05-01-preview

Headers

Parameters/Headers Value/Description
name The name of your diagnostic setting.
resourceUri subscriptions/{SUBSCRIPTION_ID}/resourceGroups/{RESOURCE_GROUP}/providers/Microsoft.Cache/Redis/{CACHE_NAME}
api-version 2017-05-01-preview
Content-Type application/json

Body

{
    "properties": {
      "storageAccountId": "/subscriptions/df602c9c-7aa0-407d-a6fb-eb20c8bd1192/resourceGroups/apptest/providers/Microsoft.Storage/storageAccounts/appteststorage1",
      "eventHubAuthorizationRuleId": "/subscriptions/1a66ce04-b633-4a0b-b2bc-a912ec8986a6/resourceGroups/montest/providers/microsoft.eventhub/namespaces/mynamespace/eventhubs/myeventhub/authorizationrules/myrule",
      "eventHubName": "myeventhub",
      "workspaceId": "/subscriptions/4b9e8510-67ab-4e9a-95a9-e2f1e570ea9c/resourceGroups/insights-integration/providers/Microsoft.OperationalInsights/workspaces/myworkspace",
      "logs": [
        {
          "category": "ConnectedClientList",
          "enabled": true,
          "retentionPolicy": {
            "enabled": false,
            "days": 0
          }
        }
      ]
    }
}

Enable Connection Logging using Azure CLI

Use the az monitor diagnostic-settings create command to create a diagnostic setting with the Azure CLI. For more for information on command and parameter descriptions, see Create diagnostic settings to send platform logs and metrics to different destinations. This example shows how to use the Azure CLI to stream data to four different endpoints:

az monitor diagnostic-settings create 
    --resource /subscriptions/{subscriptionID}/resourceGroups/{resourceGroupname}/providers/Microsoft.Cache/Redis/{cacheName}
    --name {logName}
    --logs '[{"category": "ConnectedClientList","enabled": true,"retentionPolicy": {"enabled": false,"days": 0}}]'    
    --event-hub {eventHubName}
    --event-hub-rule /subscriptions/{subscriptionID}/resourceGroups/{resourceGroupName}/providers/microsoft.eventhub/namespaces/{eventHubNamespace}/authorizationrule/{ruleName}
    --storage-account /subscriptions/{subscriptionID}/resourceGroups/{resourceGroupName}/providers/Microsoft.Storage/storageAccounts/{storageAccountName}
    --workspace /subscriptions/{subscriptionID}/resourceGroups/{resourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{logAnalyticsWorkspaceName}
    --marketplace-partner-id/subscriptions/{subscriptionID}/resourceGroups{resourceGroupname}/proviers/Microsoft.Datadog/monitors/mydatadog

Contents of the Connection Logs

These fields and properties appear in the ConnectedClientList log category. In Azure Monitor, logs are collected in the ACRConnectedClientList table under the resource provider name of MICROSOFT.CACHE.

Azure Storage field or property Azure Monitor Logs property Description
time TimeGenerated The timestamp of when the log was generated in UTC.
location Location The location (region) the Azure Cache for Redis instance was accessed in.
category n/a Available log categories: ConnectedClientList.
resourceId _ResourceId The Azure Cache for Redis resource for which logs are enabled.
operationName OperationName The Redis operation associated with the log record.
properties n/a The contents of this field are described in the rows that follow.
tenant CacheName The name of the Azure Cache for Redis instance.
roleInstance RoleInstance The role instance that logged the client list.
connectedClients.ip ClientIp The Redis client IP address.
connectedClients.privateLinkIpv6 PrivateLinkIpv6 The Redis client private link IPv6 address (if applicable).
connectedClients.count ClientCount The number of Redis client connections from the associated IP address.

Sample storage account log

If you send your logs to a storage account, the contents of the logs look like this.

{
    "time": "2021-08-05T21:04:58.0466086Z",
    "location": "canadacentral",
    "category": "ConnectedClientList",
    "properties": {
        "tenant": "mycache", 
        "connectedClients": [
            {
                "ip": "192.123.43.36", 
                "count": 86
            },
            {
                "ip": "10.1.1.4",
                "privateLinkIpv6": "fd40:8913:31:6810:6c31:200:a01:104", 
                "count": 1
            }
        ],
        "roleInstance": "1"
    },
    "resourceId": "/SUBSCRIPTIONS/E6761CE7-A7BC-442E-BBAE-950A121933B5/RESOURCEGROUPS/AZURE-CACHE/PROVIDERS/MICROSOFT.CACHE/REDIS/MYCACHE", 
    "Level": 4,
    "operationName": "Microsoft.Cache/ClientList"
}

Log Analytics Queries

Note

For a tutorial on how to use Azure Log Analytics, see Overview of Log Analytics in Azure Monitor. Remember that it may take up to 90 minutes before logs show up in Log Analtyics.

Here are some basic queries to use as models.

  • Azure Cache for Redis client connections per hour within the specified IP address range:
let IpRange = "10.1.1.0/24";
ACRConnectedClientList
// For particular datetime filtering, add '| where TimeGenerated between (StartTime .. EndTime)'
| where ipv4_is_in_range(ClientIp, IpRange)
| summarize ConnectionCount = sum(ClientCount) by TimeRange = bin(TimeGenerated, 1h)
  • Unique Redis client IP addresses that have connected to the cache:
ACRConnectedClientList
| summarize count() by ClientIp

Next steps

For detailed information about how to create a diagnostic setting by using the Azure portal, CLI, or PowerShell, see create diagnostic setting to collect platform logs and metrics in Azure article.