Edit

How to log events to Azure Event Hubs in Azure API Management

  • Article
  • 3 minutes to read

Azure Event Hubs is a highly scalable data ingress service that can ingest millions of events per second so that you can process and analyze the massive amounts of data produced by your connected devices and applications. Event Hubs acts as the "front door" for an event pipeline, and once data is collected into an event hub, it can be transformed and stored using any real-time analytics provider or batching/storage adapters. Event Hubs decouples the production of a stream of events from the consumption of those events, so that event consumers can access the events on their own schedule.

This article describes how to log API Management events using Azure Event Hubs.

Create an Azure Event Hub

For detailed steps on how to create an event hub and get connection strings that you need to send and receive events to and from the Event Hub, see Create an Event Hubs namespace and an event hub using the Azure portal.

Note

The Event Hub resource can be in a different subscription or even a different tenant than the API Management resource

Create an API Management logger

Now that you have an Event Hub, the next step is to configure a Logger in your API Management service so that it can log events to the Event Hub.

API Management loggers are configured using the API Management REST API. For detailed request examples, see how to create Loggers.

Configure log-to-eventhub policies

Once your logger is configured in API Management, you can configure your log-to-eventhub policy to log the desired events. The log-to-eventhub policy can be used in either the inbound policy section or the outbound policy section.

  1. Browse to your APIM instance.
  2. Select the API tab.
  3. Select the API to which you want to add the policy. In this example, we're adding a policy to the Echo API in the Unlimited product.
  4. Select All operations.
  5. On the top of the screen, select the Design tab.
  6. In the Inbound or Outbound processing window, click the triangle (next to the pencil).
  7. Select the Code editor. For more information, see How to set or edit policies.
  8. Position your cursor in the inbound or outbound policy section.
  9. In the window on the right, select Advanced policies > Log to EventHub. This inserts the log-to-eventhub policy statement template.
<log-to-eventhub logger-id="logger-id">
    @{
        return new JObject(
            new JProperty("EventTime", DateTime.UtcNow.ToString()),
            new JProperty("ServiceName", context.Deployment.ServiceName),
            new JProperty("RequestId", context.RequestId),
            new JProperty("RequestIp", context.Request.IpAddress),
            new JProperty("OperationName", context.Operation.Name)
        ).ToString();
    }
</log-to-eventhub>

Replace logger-id with the value you used for {loggerId} in the request URL to create the logger in the previous step.

You can use any expression that returns a string as the value for the log-to-eventhub element. In this example, a string in JSON format containing the date and time, service name, request ID, request IP address, and operation name is logged.

Click Save to save the updated policy configuration. As soon as it is saved the policy is active and events are logged to the designated Event Hub.

Note

The maximum supported message size that can be sent to an event hub from this API Management policy is 200 kilobytes (KB). If a message that is sent to an event hub is larger than 200 KB, it will be automatically truncated, and the truncated message will be transferred to event hubs.

Preview the log in Event Hubs by using Azure Stream Analytics

You can preview the log in Event Hubs by using Azure Stream Analytics queries.

  1. In the Azure portal, browse to the event hub that the logger sends events to.
  2. Under Features, select the Process data tab.
  3. On the Enable real time insights from events card, select Explore.
  4. You should be able to preview the log on the Input preview tab. If the data shown isn't current, select Refresh to see the latest events.

Next steps