Monitoring Azure Event Hubs data reference
See Monitoring Azure Event Hubs for details on collecting and analyzing monitoring data for Azure Event Hubs.
Note
Azure Monitor doesn't include dimensions in the exported metrics data, that's sent to a destination like Azure Storage, Azure Event Hubs, Log Analytics, etc.
Metrics
This section lists all the automatically collected platform metrics collected for Azure Event Hubs. The resource provider for these metrics is Microsoft.EventHub/clusters or Microsoft.EventHub/namespaces.
Request metrics
Counts the number of data and management operations requests.
| Metric Name | Exportable via diagnostic settings | Unit | Aggregation type | Description | Dimensions |
|---|---|---|---|---|---|
| Incoming Requests | Yes | Count | Count | The number of requests made to the Event Hubs service over a specified period. This metric includes all the data and management plane operations. | Entity name |
| Successful Requests | No | Count | Count | The number of successful requests made to the Event Hubs service over a specified period. | Entity name Operation Result |
| Throttled Requests | No | Count | Count | The number of requests that were throttled because the usage was exceeded. | Entity name Operation Result |
The following two types of errors are classified as user errors:
- Client-side errors (In HTTP that would be 400 errors).
- Errors that occur while processing messages.
Message metrics
| Metric Name | Exportable via diagnostic settings | Unit | Aggregation type | Description | Dimensions |
|---|---|---|---|---|---|
| Incoming Messages | Yes | Count | Count | The number of events or messages sent to Event Hubs over a specified period. | Entity name |
| Outgoing Messages | Yes | Count | Count | The number of events or messages received from Event Hubs over a specified period. | Entity name |
| Captured Messages | No | Count | Count | The number of captured messages. | Entity name |
| Incoming Bytes | Yes | Bytes | Count | Incoming bytes for an event hub over a specified period. | Entity name |
| Outgoing Bytes | Yes | Bytes | Count | Outgoing bytes for an event hub over a specified period. | Entity name |
| Size | No | Bytes | Average | Size of an event hub in bytes. | Entity name |
Note
- These values are point-in-time values. Incoming messages that were consumed immediately after that point-in-time may not be reflected in these metrics.
- The Incoming requests metric includes all the data and management plane operations. The Incoming messages metric gives you the total number of events that are sent to the event hub. For example, if you send a batch of 100 events to an event hub, it'll count as 1 incoming request and 100 incoming messages.
Capture metrics
| Metric Name | Exportable via diagnostic settings | Unit | Aggregation type | Description | Dimensions |
|---|---|---|---|---|---|
| Captured Messages | No | Count | Count | The number of captured messages. | Entity name |
| Captured Bytes | No | Bytes | Count | Captured bytes for an event hub | Entity name |
| Capture Backlog | No | Count | Count | Capture backlog for an event hub | Entity name |
Connection metrics
| Metric Name | Exportable via diagnostic settings | Unit | Aggregation type | Description | Dimensions |
|---|---|---|---|---|---|
| Active Connections | No | Count | Average | The number of active connections on a namespace and on an entity (event hub) in the namespace. Value for this metric is a point-in-time value. Connections that were active immediately after that point-in-time may not be reflected in the metric. | Entity name |
| Connections Opened | No | Count | Average | The number of open connections. | Entity name |
| Connections Closed | No | Count | Average | The number of closed connections. | Entity name |
Error metrics
| Metric Name | Exportable via diagnostic settings | Unit | Aggregation type | Description | Dimensions |
|---|---|---|---|---|---|
| Server Errors | No | Count | Count | The number of requests not processed because of an error in the Event Hubs service over a specified period. | Entity name Operation Result |
| User Errors | No | Count | Count | The number of requests not processed because of user errors over a specified period. | Entity name Operation Result |
| Quota Exceeded Errors | No | Count | Count | The number of errors caused by exceeding quotas over a specified period. | Entity name Operation Result |
Note
Logic Apps creates epoch receivers and receivers may be moved from one node to another depending on the service load. During those moves, ReceiverDisconnection exceptions may occur. They are counted as user errors on the Event Hubs service side. Logic Apps may collect failures from Event Hubs clients so that you can view them in user logs.
Metric dimensions
Azure Event Hubs supports the following dimensions for metrics in Azure Monitor. Adding dimensions to your metrics is optional. If you don't add dimensions, metrics are specified at the namespace level.
| Dimension name | Description |
|---|---|
| Entity Name | Name of the event hub. With the 'Incoming Requests' metric, the Entity Name dimension has a value of '-NamespaceOnlyMetric-' in addition to all your event hubs. It represents the requests that were made at the namespace level. Examples include a request to list all event hubs in the namespace or requests to entities that failed authentication or authorization. |
Resource logs
Azure Event Hubs now has the capability to dispatch logs to either of two destination tables - Azure Diagnostic or Resource specific tables in Log Analytics. You could use the toggle available on Azure portal to choose destination tables.
Event Hubs captures diagnostic logs for the following categories:
| Category | Description |
|---|---|
| Archive Logs | Captures information about Event Hubs Capture operations, specifically, logs related to capture errors. |
| Operational Logs | Capture all management operations that are performed on the Azure Event Hubs namespace. Data operations aren't captured, because of the high volume of data operations that are conducted on Azure Event Hubs. |
| Auto scale logs | Captures autoinflate operations done on an Event Hubs namespace. |
| Kafka coordinator logs | Captures Kafka coordinator operations related to Event Hubs. |
| Kafka user error logs | Captures information about Kafka APIs called on Event Hubs. |
| Event Hubs virtual network connection event | Captures information about IP addresses and virtual networks sending traffic to Event Hubs. |
| Customer-managed key user logs | Captures operations related to customer-managed key. |
| Runtime Audit Logs | Capture aggregated diagnostic information for all data plane access operations (such as send or receive events) in Event Hubs. |
| Application Metric Logs | Capture the aggregated information on certain metrics related to data plane operations. |
All logs are stored in JavaScript Object Notation (JSON) format. Each entry has string fields that use the format described in the following sections.
Archive logs schema
Archive log JSON strings include elements listed in the following table:
| Name | Description | Supported in Azure Diagnostics | Supported in AZMSArchiveLogs (Resource specific table) |
|---|---|---|---|
TaskName |
Description of the task that failed | Yes | Yes |
ActivityId |
Internal ID, used for tracking | Yes | Yes |
trackingId |
Internal ID, used for tracking | Yes | Yes |
resourceId |
Azure Resource Manager resource ID | yes | Yes |
eventHub |
Event hub full name (includes namespace name) | Yes | No |
EventhubName |
Name of event hub entity | No | Yes |
partitionId |
Event hub's partition being written to | Yes | Yes |
archiveStep |
possible values: ArchiveFlushWriter, DestinationInit | Yes | Yes |
startTime |
Failure start time | Yes | No |
Time Generated (UTC) |
Timestamp of operation | No | Yes |
failures |
Number of times the failure occurred | Yes | Yes |
durationInSeconds |
Duration of failure | Yes | Yes |
message |
Error message | Yes | Yes |
category |
Log Category | Yes | No |
Provider |
Name of the service emitting the logs, for example, Event Hubs | No | Yes |
Type |
Type of log emitted | No | Yes |
The following code is an example of an archive log JSON string:
AzureDiagnostics:
{
"TaskName": "EventHubArchiveUserError",
"ActivityId": "000000000-0000-0000-0000-0000000000000",
"trackingId": "0000000-0000-0000-0000-00000000000000000",
"resourceId": "/SUBSCRIPTIONS/000000000-0000-0000-0000-0000000000000/RESOURCEGROUPS/<Resource Group Name>/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/<Event Hubs Namespace Name>",
"eventHub": "<Event Hub full name>",
"partitionId": "1",
"archiveStep": "ArchiveFlushWriter",
"startTime": "9/22/2016 5:11:21 AM",
"failures": 3,
"durationInSeconds": 360,
"message": "Microsoft.WindowsAzure.Storage.StorageException: The remote server returned an error: (404) Not Found. ---> System.Net.WebException: The remote server returned an error: (404) Not Found.\r\n at Microsoft.WindowsAzure.Storage.Shared.Protocol.HttpResponseParsers.ProcessExpectedStatusCodeNoException[T](HttpStatusCode expectedStatusCode, HttpStatusCode actualStatusCode, T retVal, StorageCommandBase`1 cmd, Exception ex)\r\n at Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob.<PutBlockImpl>b__3e(RESTCommand`1 cmd, HttpWebResponse resp, Exception ex, OperationContext ctx)\r\n at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.EndGetResponse[T](IAsyncResult getResponseResult)\r\n --- End of inner exception stack trace ---\r\n at Microsoft.WindowsAzure.Storage.Core.Util.StorageAsyncResult`1.End()\r\n at Microsoft.WindowsAzure.Storage.Core.Util.AsyncExtensions.<>c__DisplayClass4.<CreateCallbackVoid>b__3(IAsyncResult ar)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.",
"category": "ArchiveLogs"
}
Resource specific table entry:
{
"TaskName": "EventHubArchiveUserError",
"ActivityId": "000000000-0000-0000-0000-0000000000000",
"trackingId": "0000000-0000-0000-0000-00000000000000000",
"resourceId": "/SUBSCRIPTIONS/000000000-0000-0000-0000-0000000000000/RESOURCEGROUPS/<Resource Group Name>/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/<Event Hubs Namespace Name>",
"EventHubName": "<Event Hub full name>",
"partitionId": "1",
"archiveStep": "ArchiveFlushWriter",
"TimeGenerated(UTC)": "9/22/2016 5:11:21 AM",
"failures": 3,
"durationInSeconds": 360,
"message": "Microsoft.WindowsAzure.Storage.StorageException: The remote server returned an error: (404) Not Found. ---> System.Net.WebException: The remote server returned an error: (404) Not Found.\r\n at Microsoft.WindowsAzure.Storage.Shared.Protocol.HttpResponseParsers.ProcessExpectedStatusCodeNoException[T](HttpStatusCode expectedStatusCode, HttpStatusCode actualStatusCode, T retVal, StorageCommandBase`1 cmd, Exception ex)\r\n at Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob.<PutBlockImpl>b__3e(RESTCommand`1 cmd, HttpWebResponse resp, Exception ex, OperationContext ctx)\r\n at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.EndGetResponse[T](IAsyncResult getResponseResult)\r\n --- End of inner exception stack trace ---\r\n at Microsoft.WindowsAzure.Storage.Core.Util.StorageAsyncResult`1.End()\r\n at Microsoft.WindowsAzure.Storage.Core.Util.AsyncExtensions.<>c__DisplayClass4.<CreateCallbackVoid>b__3(IAsyncResult ar)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.",
"Provider":"EVENTHUB",
"Type":"AZMSArchiveLogs"
}
Operational logs schema
Operational log JSON strings include elements listed in the following table:
| Name | Description | Supported in AzureDiagnostics | Supported in AZMSOperationalLogs (Resource specific table) |
|---|---|---|---|
ActivityId |
Internal ID, used for tracking purposes | Yes | Yes |
EventName |
Operation name. For a list of values for this element, see the Event names | Yes | Yes |
resourceId |
Azure Resource Manager resource ID | Yes | Yes |
SubscriptionId |
Subscription ID | Yes | Yes |
EventTimeString |
Operation time | Yes | No |
Time Generated (UTC) |
Timestamp of operation | No | Yes |
EventProperties |
Properties for the operation. This element provides more information about the event as shown in the following example. | Yes | Yes |
Status |
Operation status. The value can be either Succeeded or Failed. | Yes | Yes |
Caller |
Caller of operation (Azure portal or management client) | Yes | Yes |
Category |
Log Category | Yes | No |
Provider |
Name of the service emitting the logs, for example, Event Hubs | No | Yes |
Type |
Type of logs emitted | No | Yes |
The following code is an example of an operational log JSON string:
AzureDiagnostics:
Example:
{
"ActivityId": "00000000-0000-0000-0000-00000000000000",
"EventName": "Create EventHub",
"resourceId": "/SUBSCRIPTIONS/00000000-0000-0000-0000-0000000000000/RESOURCEGROUPS/<Resource Group Name>/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/<Event Hubs namespace name>",
"SubscriptionId": "000000000-0000-0000-0000-000000000000",
"EventTimeString": "9/28/2016 8:40:06 PM +00:00",
"EventProperties": "{\"SubscriptionId\":\"0000000000-0000-0000-0000-000000000000\",\"Namespace\":\"<Namespace Name>\",\"Via\":\"https://<Namespace Name>.servicebus.windows.net/f8096791adb448579ee83d30e006a13e/?api-version=2016-07\",\"TrackingId\":\"5ee74c9e-72b5-4e98-97c4-08a62e56e221_G1\"}",
"Status": "Succeeded",
"Caller": "ServiceBus Client",
"category": "OperationalLogs"
}
Resource specific table entry:
Example:
{
"ActivityId": "00000000-0000-0000-0000-00000000000000",
"EventName": "Create EventHub",
"resourceId": "/SUBSCRIPTIONS/00000000-0000-0000-0000-0000000000000/RESOURCEGROUPS/<Resource Group Name>/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/<Event Hubs namespace name>",
"SubscriptionId": "000000000-0000-0000-0000-000000000000",
"TimeGenerated (UTC)": "9/28/2016 8:40:06 PM +00:00",
"EventProperties": "{\"SubscriptionId\":\"0000000000-0000-0000-0000-000000000000\",\"Namespace\":\"<Namespace Name>\",\"Via\":\"https://<Namespace Name>.servicebus.windows.net/f8096791adb448579ee83d30e006a13e/?api-version=2016-07\",\"TrackingId\":\"5ee74c9e-72b5-4e98-97c4-08a62e56e221_G1\"}",
"Status": "Succeeded",
"Caller": "ServiceBus Client",
"Provider": "EVENTHUB",
"Type":"AZMSOperationalLogs"
}
Event names
Event name is populated as operation type + resource type from the following enumerations. For example, Create Queue, Retrieve Event Hub, or Delete Rule.
| Operation type | Resource type |
|---|---|
| - Create - Update - Delete - Retrieve - Unknown |
- Namespace - Queue - Topic - Subscription - Event Hubs - SharedAccessPolicy - UsageCredit - Rule - ConsumerGroup |
Autoscale logs schema
Autoscale log JSON includes elements listed in the following table:
| Name | Description | Supported in Azure Diagnostics | Supported in AZMSAutoscaleLogs (Resource specific table) |
|---|---|---|---|
TrackingId |
Internal ID, which is used for tracing purposes | Yes | Yes |
ResourceId |
Azure Resource Manager resource ID. | Yes | Yes |
Message |
Informational message, which provides details about autoinflate action. The message contains previous and current value of throughput unit for a given namespace and what triggered the inflate of the TU. | Yes | Yes |
Time Generated (UTC) |
Timestamp of operation | No | Yes |
Provider |
Name of Service emitting the logs, for example, Event Hubs | No | Yes |
Type |
Type of logs emitted | No | Yes |
Here's an example autoscale event:
AzureDiagnostics:
{
"TrackingId": "fb1b3676-bb2d-4b17-85b7-be1c7aa1967e",
"Message": "Scaled-up EventHub TUs (UpdateStartTimeUTC: 5/13/2021 7:48:36 AM, PreviousValue: 1, UpdatedThroughputUnitValue: 2, AutoScaleReason: 'IncomingMessagesPerSecond reached 2170')",
"ResourceId": "/subscriptions/0000000-0000-0000-0000-000000000000/resourcegroups/testrg/providers/microsoft.eventhub/namespaces/namespace-name"
}
Resource specific table entry:
{
"TrackingId": "fb1b3676-bb2d-4b17-85b7-be1c7aa1967e",
"Message": "Scaled-up EventHub TUs (UpdateStartTimeUTC: 5/13/2021 7:48:36 AM, PreviousValue: 1, UpdatedThroughputUnitValue: 2, AutoScaleReason: 'IncomingMessagesPerSecond reached 2170')",
"ResourceId": "/subscriptions/0000000-0000-0000-0000-000000000000/resourcegroups/testrg/providers/microsoft.eventhub/namespaces/namespace-name",
"timeGenerated (UTC) : "9/28/2022 8:40:06 PM +00:00",
"Provider" : "EVENTHUB",
"Type" : "AZMSAutoscaleLogs"
}
Kafka coordinator logs schema
Kafka coordinator log JSON includes elements listed in the following table:
| Name | Description | Supported in Azure Diagnostics | Supported in AZMSKafkaCoordinatorLogs (Resource specific table) |
|---|---|---|---|
RequestId |
Request ID, which is used for tracing purposes | Yes | Yes |
ResourceId |
Azure Resource Manager resource ID | Yes | Yes |
Operation |
Name of the operation done during the group coordination | Yes | Yes |
ClientId |
Client ID | Yes | Yes |
NamespaceName |
Namespace name | Yes | Yes |
SubscriptionId |
Azure subscription ID | Yes | Yes |
Message |
Informational or warning message, which provides details about actions done during the group coordination. | Yes | Yes |
Time Generated (UTC) |
Timestamp of operation | No | Yes |
Provider |
Name of Service emitting the logs, for example, ServiceBus | No | Yes |
Type |
Type of log emitted | No | Yes |
Example
AzureDiagnostics:
{
"RequestId": "FE01001A89E30B020000000304620E2A_KafkaExampleConsumer#0",
"Operation": "Join.Start",
"ClientId": "KafkaExampleConsumer#0",
"Message": "Start join group for new member namespace-name:c:$default:I:KafkaExampleConsumer#0-cc40856f7f3c4607915a571efe994e82, current group size: 0, API version: 2, session timeout: 10000ms, rebalance timeout: 300000ms.",
"SubscriptionId": "0000000-0000-0000-0000-000000000000",
"NamespaceName": "namespace-name",
"ResourceId": "/subscriptions/0000000-0000-0000-0000-000000000000/resourcegroups/testrg/providers/microsoft.eventhub/namespaces/namespace-name",
"Category": "KafkaCoordinatorLogs"
}
Resource Specific table entry:
{
"RequestId": "FE01001A89E30B020000000304620E2A_KafkaExampleConsumer#0",
"Operation": "Join.Start",
"ClientId": "KafkaExampleConsumer#0",
"Message": "Start join group for new member namespace-name:c:$default:I:KafkaExampleConsumer#0-cc40856f7f3c4607915a571efe994e82, current group size: 0, API version: 2, session timeout: 10000ms, rebalance timeout: 300000ms.",
"SubscriptionId": "0000000-0000-0000-0000-000000000000",
"NamespaceName": "namespace-name",
"ResourceId": "/subscriptions/0000000-0000-0000-0000-000000000000/resourcegroups/testrg/providers/microsoft.eventhub/namespaces/namespace-name",
"Time Generated (UTC) ": "9/28/2022 8:40:06 PM +00:00",
"Provider" : "EVENTHUB",
"Type" : "AZMSKafkaCoordinatorLogs"
}
Kafka user error logs schema
Kafka user error log JSON includes elements listed in the following table:
| Name | Description | Supported in Azure Diagnostics | Supported in AZMSKafkaUserErrorLogs (Resource specific table) |
|---|---|---|---|
TrackingId |
Tracking ID, which is used for tracing purposes. | Yes | Yes |
NamespaceName |
Namespace name | Yes | Yes |
Eventhub |
Event hub name | Yes | Yes |
PartitionId |
Partition ID | Yes | Yes |
GroupId |
Group ID | Yes | Yes |
ClientId |
Client ID | Yes | Yes |
ResourceId |
Azure Resource Manager resource ID. | Yes | Yes |
Message |
Informational message, which provides details about an error | Yes | Yes |
TimeGenerated (UTC) |
Timestamp for executed operation | No | Yes |
Provider |
Name of service emitting the logs, for example, Event Hubs | No | Yes |
Type |
Type of log emitted | NO | Yes |
Event Hubs virtual network connection event schema
Event Hubs virtual network (virtual network) connection event JSON includes elements listed in the following table:
| Name | Description | Supported in Azure Diagnostics | Supported in AZMSVNetConnectionevents (Resource specific table) |
|---|---|---|---|
SubscriptionId |
Azure subscription ID | Yes | Yes |
NamespaceName |
Namespace name | Yes | Yes |
IPAddress |
IP address of a client connecting to the Event Hubs service | Yes | Yes |
Action |
Action done by the Event Hubs service when evaluating connection requests. Supported actions are Accept Connection and Deny Connection. | Yes | Yes |
Reason |
Provides a reason why the action was done | Yes | No |
Message |
Provides a reason why the action was done | No | Yes |
Count |
Number of occurrences for the given action | Yes | Yes |
ResourceId |
Azure Resource Manager resource ID. | Yes | Yes |
Time Generated (UTC) |
Timestamp of operation | No | Yes |
Provider |
Name of Service emitting the logs, for example, ServiceBus | No | Yes |
Type |
AZMSVNetConnectionevents | No | Yes |
Virtual network logs are generated only if the namespace allows access from selected networks or from specific IP addresses (IP filter rules). If you don't want to restrict the access to your namespace using these features and still want to get virtual network logs to track IP addresses of clients connecting to the Event Hubs namespace, you could use the following workaround. Enable IP filtering, and add the total addressable IPv4 range (0.0.0.0/1 - 128.0.0.0/1) and IPv6 range (::/1 - 8000::/1). Event Hubs IP filtering doesn't support IPv6 ranges. You might see private endpoint addresses in the IPv6 format in the log.
Example
AzureDiagnostics:
{
"SubscriptionId": "0000000-0000-0000-0000-000000000000",
"NamespaceName": "namespace-name",
"IPAddress": "1.2.3.4",
"Action": "Deny Connection",
"Reason": "IPAddress doesn't belong to a subnet with Service Endpoint enabled.",
"Count": "65",
"ResourceId": "/subscriptions/0000000-0000-0000-0000-000000000000/resourcegroups/testrg/providers/microsoft.eventhub/namespaces/namespace-name",
"Category": "EventHubVNetConnectionEvent"
}
Resource specific table entry:
{
"SubscriptionId": "0000000-0000-0000-0000-000000000000",
"NamespaceName": "namespace-name",
"IPAddress": "1.2.3.4",
"Action": "Deny Connection",
"Message": "IPAddress doesn't belong to a subnet with Service Endpoint enabled.",
"Count": "65",
"ResourceId": "/subscriptions/0000000-0000-0000-0000-000000000000/resourcegroups/testrg/providers/microsoft.eventhub/namespaces/namespace-name",
"Provider": "EVENTHUB",
"Time Generated (UTC) ": "9/28/2022 8:40:06 PM +00:00",
"Type" : "AZMSKafkauserErrorlogs"
}
Customer-managed key user logs schema
Customer-managed key user log JSON includes elements listed in the following table:
| Name | Description | Supported in Azure Diagnostics | Supported in AZMSCustomerManagedKeyUserLogs (Resource specific table) |
|---|---|---|---|
Category |
Type of category for a message. It's one of the following values: error and info. For example, if the key from your key vault is being disabled, then it would be an information category or if a key can't be unwrapped, it could fall under error. | Yes | Yes |
ResourceId |
Internal resource ID, which includes Azure subscription ID and namespace name | Yes | Yes |
KeyVault |
Name of the Key Vault resource | Yes | Yes |
Key |
Name of the Key Vault key that's used to encrypt the Event Hubs namespace. | Yes | Yes |
Version |
Version of the Key Vault key. | Yes | Yes |
Operation |
The operation that's performed on the key in your key vault. For example, disable/enable the key, wrap, or unwrap. | Yes | Yes |
Code |
The code associated with the operation. Example: Error code, 404 means that key wasn't found. | Yes | Yes |
Message |
Message, which provides details about an error or informational message | Yes | Yes |
Time Generated (UTC) |
Timestamp of operation | No | Yes |
Provider |
Name of Service emitting the logs, for example, ServiceBus | No | Yes |
Type |
Type of log emitted | No | Yes |
Here's an example of the log for a customer managed key:
AzureDiagnostics:
{
"TaskName": "CustomerManagedKeyUserLog",
"ActivityId": "11111111-1111-1111-1111-111111111111",
"category": "error"
"resourceId": "/SUBSCRIPTIONS/11111111-1111-1111-1111-11111111111/RESOURCEGROUPS/DEFAULT-EVENTHUB-CENTRALUS/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/FBETTATI-OPERA-EVENTHUB",
"keyVault": "https://mykeyvault.vault-int.azure-int.net",
"key": "mykey",
"version": "1111111111111111111111111111111",
"operation": "wrapKey",
"code": "404",
"message": "Key not found: ehbyok0/111111111111111111111111111111"
}
{
"TaskName": "CustomerManagedKeyUserLog",
"ActivityId": "11111111111111-1111-1111-1111111111111",
"category": "info"
"resourceId": "/SUBSCRIPTIONS/111111111-1111-1111-1111-11111111111/RESOURCEGROUPS/DEFAULT-EVENTHUB-CENTRALUS/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/FBETTATI-OPERA-EVENTHUB",
"keyVault": "https://mykeyvault.vault-int.azure-int.net",
"key": "mykey",
"version": "111111111111111111111111111111",
"operation": "disable | restore",
"code": "",
"message": ""
}
Resource specific table entry:
{
"TaskName": "CustomerManagedKeyUserLog",
"ActivityId": "11111111-1111-1111-1111-111111111111",
"category": "error"
"resourceId": "/SUBSCRIPTIONS/11111111-1111-1111-1111-11111111111/RESOURCEGROUPS/DEFAULT-EVENTHUB-CENTRALUS/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/FBETTATI-OPERA-EVENTHUB",
"keyVault": "https://mykeyvault.vault-int.azure-int.net",
"key": "mykey",
"version": "1111111111111111111111111111111",
"operation": "wrapKey",
"code": "404",
"message": "Key not found: ehbyok0/111111111111111111111111111111",
"Provider": "EVENTHUB",
"Time Generated (UTC) ": "9/28/2022 8:40:06 PM +00:00",
"Type" : "AZMSCustomerManagedKeyUserLogs"
}
{
"TaskName": "CustomerManagedKeyUserLog",
"ActivityId": "11111111111111-1111-1111-1111111111111",
"category": "info"
"resourceId": "/SUBSCRIPTIONS/111111111-1111-1111-1111-11111111111/RESOURCEGROUPS/DEFAULT-EVENTHUB-CENTRALUS/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/FBETTATI-OPERA-EVENTHUB",
"keyVault": "https://mykeyvault.vault-int.azure-int.net",
"key": "mykey",
"version": "111111111111111111111111111111",
"operation": "disable | restore",
"code": "",
"message": "",
"Provider": "EVENTHUB",
"Time Generated (UTC) ": "9/28/2022 8:40:06 PM +00:00",
"Type" : "AZMSCustomerManagedKeyUserLogs"
}
Following are the common errors codes to look for when BYOK encryption is enabled.
| Action | Error code | Resulting state of data |
|---|---|---|
| Remove wrap/unwrap permission from a key vault | 403 | Inaccessible |
| Remove Microsoft Entra ID role membership from a Microsoft Entra principal that granted the wrap/unwrap permission | 403 | Inaccessible |
| Delete an encryption key from the key vault | 404 | Inaccessible |
| Delete the key vault | 404 | Inaccessible (assumes soft-delete is enabled, which is a required setting.) |
| Changing the expiration period on the encryption key such that it's already expired | 403 | Inaccessible |
| Changing the NBF (not before) such that key encryption key isn't active | 403 | Inaccessible |
| Selecting the Allow MSFT Services option for the key vault firewall or otherwise blocking network access to the key vault that has the encryption key | 403 | Inaccessible |
| Moving the key vault to a different tenant | 404 | Inaccessible |
| Intermittent network issue or DNS/AAD/MSI outage | Accessible using cached data encryption key |
Runtime audit logs
Runtime audit logs capture aggregated diagnostic information for all data plane access operations (such as send or receive events) in Event Hubs.
Note
Runtime audit logs are available only in premium and dedicated tiers.
Runtime audit logs include the elements listed in the following table:
| Name | Description | Supported in Azure Diagnostics | Supported in Resource Specific table |
|---|---|---|---|
ActivityId |
A randomly generated UUID that ensures uniqueness for the audit activity. | Yes | Yes |
ActivityName |
Runtime operation name. | Yes | Yes |
ResourceId |
Resource associated with the activity. | Yes | Yes |
Timestamp |
Aggregation time. | Yes | No |
TimeGenerated [UTC] |
Time of executed operation (in UTC) | No | Yes |
Status |
Status of the activity (success or failure). | Yes | Yes |
Protocol |
Type of the protocol associated with the operation. | Yes | Yes |
AuthType |
Type of authentication (Azure Active Directory or SAS Policy). | Yes | Yes |
AuthKey |
Azure Active Directory application ID or SAS policy name that's used to authenticate to a resource. | Yes | Yes |
NetworkType |
Type of the network access: Public or Private. |
Yes | Yes |
ClientIP |
IP address of the client application. | Yes | Yes |
Count |
Total number of operations performed during the aggregated period of 1 minute. | Yes | Yes |
Properties |
Metadata that are specific to the data plane operation. | Yes | Yes |
Category |
Log category | Yes | NO |
Provider |
Name of Service emitting the logs e.g., Eventhub | No | Yes |
Type |
Type of logs emitted | No | Yes |
Here's an example of a runtime audit log entry:
AzureDiagnostics :
{
"ActivityId": "<activity id>",
"ActivityName": "ConnectionOpen | Authorization | SendMessage | ReceiveMessage",
"ResourceId": "/SUBSCRIPTIONS/xxx/RESOURCEGROUPS/<Resource Group Name>/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/<Event Hubs namespace>/eventhubs/<event hub name>",
"Time": "1/1/2021 8:40:06 PM +00:00",
"Status": "Success | Failure",
"Protocol": "AMQP | KAFKA | HTTP | Web Sockets",
"AuthType": "SAS | Azure Active Directory",
"AuthId": "<AAD application name | SAS policy name>",
"NetworkType": "Public | Private",
"ClientIp": "x.x.x.x",
"Count": 1,
"Category": "RuntimeAuditLogs"
}
Resource specific table entry:
{
"ActivityId": "<activity id>",
"ActivityName": "ConnectionOpen | Authorization | SendMessage | ReceiveMessage",
"ResourceId": "/SUBSCRIPTIONS/xxx/RESOURCEGROUPS/<Resource Group Name>/PROVIDERS/MICROSOFT.EVENTHUB/NAMESPACES/<Event Hubs namespace>/eventhubs/<event hub name>",
"TimeGenerated (UTC)": "1/1/2021 8:40:06 PM +00:00",
"Status": "Success | Failure",
"Protocol": "AMQP | KAFKA | HTTP | Web Sockets",
"AuthType": "SAS | Azure Active Directory",
"AuthId": "<AAD application name | SAS policy name>",
"NetworkType": "Public | Private",
"ClientIp": "x.x.x.x",
"Count": 1,
"Type": "AZMSRuntimeAUditLogs",
"Provider":"EVENTHUB"
}
Application metrics logs
Application metrics logs capture the aggregated information on certain metrics related to data plane operations. The captured information includes the following runtime metrics.
Note
Application metrics logs are available only in premium and dedicated tiers.
| Name | Description |
|---|---|
ConsumerLag |
Indicate the lag between consumers and producers. |
NamespaceActiveConnections |
Details of active connections established from a client to the event hub. |
GetRuntimeInfo |
Obtain run time information from Event Hubs. |
GetPartitionRuntimeInfo |
Obtain the approximate runtime information for a logical partition of an event hub. |
IncomingMessages |
Details of number of messages published to Event Hubs. |
IncomingBytes |
Details of Publisher throughput sent to Event Hubs |
OutgoinMessages |
Details of number of messages consumed from Event Hubs. |
OutgoingBytes |
Details of Consumer throughput from Event Hubs. |
OffsetCommit |
Number of offset commit calls made to the event hub |
OffsetFetch |
Number of offset fetch calls made to the event hub. |
Azure Monitor Logs tables
Azure Event Hubs uses Kusto tables from Azure Monitor Logs. You can query these tables with Log Analytics. For a list of Kusto tables the service uses, see Azure Monitor Logs table reference.
You can view our sample queries to get started with different log categories.
Important
Dimensions aren't exported to a Log Analytics workspace.
Next steps
- For details on monitoring Azure Event Hubs, see Monitoring Azure Event Hubs.
- For details on monitoring Azure resources, see Monitoring Azure resources with Azure Monitor.
Feedback
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see: https://aka.ms/ContentUserFeedback.
Submit and view feedback for