Pull delivery with HTTP

This article builds on Introduction to Azure Event Grid and concepts articles to provide essential information before you start using Event Grid’s pull delivery over HTTP. It covers fundamental concepts, resource models, and supported message delivery modes. At the end of this document, you find useful links to articles that guide you on how to use Event Grid and to articles that offer in-depth conceptual information.

Note

This document helps you get started with Event Grid capabilities that use the HTTP protocol. This article is suitable for users who need to integrate applications on the cloud. If you need to communicate IoT device data, see Overview of the Message Queuing Telemetry and Transport (MQTT) Broker feature in Azure Event Grid.

CloudEvents

Event Grid namespace topics accept events that comply with the Cloud Native Computing Foundation (CNCF)’s open standard CloudEvents 1.0 specification by using the HTTP protocol binding with JSON format.

For more information, see CloudEvents support.

CloudEvents content modes

The CloudEvents specification defines three content modes you can use: binary, structured, and batched.

Important

With any content mode, you can exchange text (JSON, text/*, and similar types) or binary encoded event data. The binary content mode isn't exclusively used for sending binary data.

The content modes aren't about the encoding you use, binary, or text, but about how the event data and its metadata are described and exchanged. The structured content mode uses a single structure, such as a JSON object, where both the context attributes and event data are together in the HTTP payload. The binary content mode separates context attributes, which are mapped to HTTP headers, and event data, which is the HTTP payload encoded according to the media type value in Content-Type.

For more information, see CloudEvents content modes.

Messages and events

A CloudEvent typically carries event data announcing an occurrence in a system, that is, a system state change. However, you can convey any kind of data when using CloudEvents. For example, you might want to use the CloudEvents exchange format to send a command message to request an action to a downstream application. Another example is when you're routing messages from Event Grid's MQTT broker to a topic. Under this scenario, you're routing an MQTT message wrapped up in a CloudEvents envelope.

Pull delivery

With pull delivery, your application connects to Event Grid to read CloudEvents using queue-like semantics.

Pull delivery offers these event consumption benefits:

• Consume events at your own pace, at scale, or at an ingress rate your application supports.

• Consume events at a time of your own choosing. For example, given business requirements, process messages at night.

• Consume events over a private link so that your data uses private IP space.

Note

• Namespaces provide a simpler resource model featuring a single kind of topic. Currently, Event Grid supports publishing your own application events through namespace topics. You can't consume events from Azure services or partner SaaS systems by using namespace topics. You also can't create system topics, domain topics, or partner topics in a namespace.
• Namespace topics support CloudEvents JSON format.

Queue event subscriptions

When receiving events or using operations that manage event state, an application specifies a namespace HTTP endpoint, a topic name, and the name of a queue event subscription. A queue event subscription has its deliveryMode set to *queue*. Queue event subscriptions are used to consume events using the pull delivery API. For more information about how to create these resources, see create namespaces, topics, and event subscriptions.

You use an event subscription to define the filtering criteria for events. By defining the filtering criteria, you effectively define the set of events that are available for consumption through that event subscription. One or more subscriber (consumer) applications can connect to the same namespace endpoint and use the same topic and event subscription.

Pull delivery operations

Your application uses the following operations when working with pull delivery.

• A receive operation reads one or more events by using a single request to Event Grid. By default, the broker waits for up to 60 seconds for events to become available. For example, events become available for delivery when they're first published. A successful receive request returns zero or more events. If events are available, it returns as many available events as possible up to the event count requested. Event Grid also returns a lock token for every event read.
• A lock token is a kind of handle that identifies an event that you can use to control its state.
• Once a consumer application receives an event and processes it, it acknowledges that event. This operation instructs Event Grid to delete the event so it isn't redelivered to another client. The consumer application acknowledges one or more tokens with a single request by specifying their lock tokens before they expire.

In some other occasions, your consumer application might want to release or reject events.

• Your consumer application releases a received event to signal Event Grid that it isn't ready to process that event and to make it available for redelivery. It does so by calling the release operation with the lock tokens identifying the events to return back to Event Grid. Your application can control if the event should be released immediately or if a delay should be used before the event is available for redelivery.

• You can opt to reject an event if there's a condition, possibly permanent, that prevents your consumer application from processing the event. For example, a malformed message can't be successfully parsed and can be rejected. Rejected events are dead-lettered if a dead-letter destination is available. Otherwise, they're dropped.

Scope on which pull delivery operations run

When you invoke a receive, acknowledge, release, reject, or renew lock operation, those actions are performed in the context of the event subscription. For example, if you acknowledge an event, that event is no longer available through the event subscription used when calling the acknowledge action. Other event subscriptions could still have the "same" event available. That availability exists because an event subscription gets a copy of the events published. Those event copies are effectively distinct from each other across event subscriptions. Each event has its own state independent of other events.

Data shape when receiving events using pull delivery

When delivering events using pull delivery, Event Grid includes an array of objects that in turn includes the event and brokerProperties objects. The value of the event property is the CloudEvent delivered in structured content mode. The brokerProperties object contains the lock token associated to the CloudEvent delivered. The following json object is a sample response from a receive operation that returns two events:

{
    "value": [
        {
            "brokerProperties": {
                "lockToken": "CiYKJDUwNjE4QTFFLUNDODQtNDZBQy1BN0Y4LUE5QkE3NjEwNzQxMxISChDXYS23Z+5Hq754VqQjxywE",
                "deliveryCount": 2
            },
            "event": {
                "specversion": "1.0",
                "id": "A234-1234-1235",
                "source": "/mycontext",
                "time": "2018-04-05T17:31:00Z",
                "type": "com.example.someeventtype",
                "data": "some data"
            }
        },
        {
            "brokerProperties": {
                "lockToken": "CiYKJDUwNjE4QTFFLUNDODQtNDZBQy1BN0Y4LUE5QkE3NjEwNzQxMxISChDLeaL+nRJLNq3/5NXd/T0b",
                "deliveryCount": 1
            },
            "event": {
                "specversion": "1.0",
                "id": "B688-1234-1235",
                "source": "/mycontext",
                "type": "com.example.someeventtype",
                "time": "2018-04-05T17:31:00Z",
                "data": {
                    "somekey" : "value",
                    "someOtherKey" : 9
                }
            }
        }
    ]
}

Push and pull delivery

Event Grid supports push and pull event delivery using HTTP. With push delivery, you define a destination in an event subscription, a webhook, or an Azure service, to which Event Grid sends events. With pull delivery, subscriber applications connect to Event Grid to consume events. Pull delivery is supported for topics in an Event Grid namespace.

Important

Event Hubs is supported as a destination for subscriptions to namespace topics. In coming releases, Event Grid Namespaces will support all destinations currently available in Event Grid Basic along with additional destinations.

When to use push delivery vs. pull delivery

The following are general guidelines to help you decide when to use pull or push delivery.

Pull delivery

• You need full control as to when to receive events. For example, your application might not be up all the time, not stable enough, or you process data at certain times.
• You need full control over event consumption. For example, a downstream service or layer in your consumer application has a problem that prevents you from processing events. In that case, the pull delivery API allows the consumer app to release an already read event back to the broker so that it can be delivered later.
• You want to use private links when receiving events, which is possible only with the pull delivery, not the push delivery.
• You don't have the ability to expose an endpoint and use push delivery, but you can connect to Event Grid to consume events.

Push delivery

• You want to avoid constant polling to determine that a system state change has occurred. You rather use Event Grid to send events to you at the time state changes happen.
• You have an application that can't make outbound calls. For example, your organization might be concerned about data exfiltration. However, your application can receive events through a public endpoint.

Next steps

The following articles provide you with information on how to use Event Grid or provide you with additional information on concepts.

• Publish and subscribe to events using Namespace Topics.

Other useful links

• Control plane and data plane Software Development Kits (SDKs).
• Data plane SDKs announcement with a plethora of information, samples, and links.
• Quotas and limits.