How to retrive APIM eventhub log from policy : log-to-eventhub

Amy Z 291 Reputation points

Hi, we've followed this post to add request/response log to eventhub then capture to storage account, it works.
Related content can be found in *.avro from storage account, but the format is not user-friendly so we try to leverage existing Apache Nifi to directly read data from eventhub.

It's weird that we can read Gatewaylogs, metrics by Nifi , but no APIM request/response in it, so we try to inspect log analytics.

Since in policy setting we log request in partition 0 and response in partition 1, we use following script to check, but nothing in 0 and 1 partition.

WHERE PartitionId = 1

Really confused how the request/response log captured in storage account, we cannot read them from log analytics and Nifi.

Here's the config about event-hub and its namespace

//Event-hub namespace
    "sku": {
        "name": "Standard",
        "tier": "Standard",
        "capacity": 5
    "id": "/subscriptions/12345678/resourceGroups/RG_xxx/providers/Microsoft.EventHub/namespaces/evhns-xxx",
    "name": "evhns-xxx",
    "type": "Microsoft.EventHub/Namespaces",
    "location": "West US 2",
    "tags": {},
    "properties": {
        "isAutoInflateEnabled": true,
        "maximumThroughputUnits": 20,
        "kafkaEnabled": true,
        "provisioningState": "Succeeded",
        "metricId": "12345678-3976-49e0-b333-389a5fdb7f7b:evhns-xxx",
        "createdAt": "2021-01-07T10:31:38.057Z",
        "updatedAt": "2021-01-07T10:32:21.077Z",
        "serviceBusEndpoint": "",
        "status": "Active"

    "id": "/subscriptions/12345678/resourceGroups/RG_xxx/providers/Microsoft.EventHub/namespaces/evhns-xxx/eventhubs/evh-xxx",
    "name": "evh-xxx",
    "type": "Microsoft.EventHub/Namespaces/EventHubs",
    "location": "West US 2",
    "properties": {
        "messageRetentionInDays": 7,
        "partitionCount": 20,
        "status": "Active",
        "createdAt": "2021-01-07T10:49:15.263",
        "updatedAt": "2021-01-18T08:28:26.973",
        "partitionIds": [
        "captureDescription": {
            "enabled": true,
            "encoding": "Avro",
            "destination": {
                "name": "EventHubArchive.AzureBlockBlob",
                "properties": {
                    "storageAccountResourceId": "/subscriptions/12345678/resourceGroups/RG_xxx/providers/Microsoft.Storage/storageAccounts/storagexxx",
                    "blobContainer": "evh-xxx",
                    "archiveNameFormat": "{Namespace}/{EventHub}/{PartitionId}/{Year}/{Month}/{Day}/{Hour}/{Minute}/{Second}"
            "intervalInSeconds": 300,
            "sizeLimitInBytes": 314572800
Azure API Management
Azure API Management
An Azure service that provides a hybrid, multi-cloud management platform for APIs.
1,796 questions
Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
566 questions
{count} votes