Receive change notifications through Azure Event Hubs

Webhooks aren't suited for receiving change notifications in high throughput scenarios or when the receiver can't expose a publicly available notification URL. As an alternative, you can use Azure Event Hubs.

Examples of high throughput scenarios where you can use Azure Event Hubs include applications subscribing to a large set of resources, applications subscribing to resources that change frequently, and multitenant applications that subscribe to resources across a large set of organizations.

The article guides you through the process of managing your Microsoft Graph subscription and how to receive change notifications through Azure Event Hubs.

Important

Authenticating Event Hubs by using shared access signatures (SAS) will be deprecated in the future. We recommend authenticating Event Hubs by using Microsoft Entra ID role-based access control (RBAC) instead.

Using Azure Event Hubs to receive change notification

Azure Event Hubs is a popular real-time events ingestion and distribution service built for scale. Using Azure Event Hubs to receive change notifications differs from webhooks in a few ways, including:

• You don't rely on publicly exposed notification URLs. The Event Hubs SDK relays the notifications to your application.
• You don't need to reply to the notification URL validation. You can ignore the validation message that you receive.
• You need to provision an event hub.
• You need to provision an Azure Key Vault or add the Microsoft Graph Change Tracking service to the Data Sender role on your event hub.

Set up the Azure Event Hubs authentication

Azure Event Hubs supports authentication through either shared access signatures (SAS) or Microsoft Entra ID role-based access control (RBAC). For more information, see Authorize access to Azure Event Hubs.

Use the Azure portal with RBAC

This section demonstrates how to set up Azure Event Hubs authentication using Microsoft Entra ID role-based access control (RBAC) on the Azure portal.

Configure the event hub

1. Sign in to the Azure portal with privileges to create resources in your Azure subscription.
2. Select Create a resource, type Event Hubs in the search bar, and then select the Event Hubs suggestion.
3. On the Event Hubs creation page, select Create.
4. Fill in the Event Hubs namespace creation details, and then select Create.
5. When the Event Hubs namespace is provisioned, go to the page for the namespace.
6. Select Event Hubs and then + Event Hub.
7. Give a name to the new event hub, and select Create.
8. After the event hub is created, go to the Event Hubs namespace, and then select Access Control (IAM) from the sidebar.
9. Select Role Assignments.
10. Select + Add and select Add Role Assignment.
11. Under Role, go to Job function roles, select Azure Event Hubs Data Sender, and then select Next.
12. Under the Members tab, select Assign access to User, group, or service principal.
13. Select + Select members, then search for and select Microsoft Graph Change Tracking.
14. Select Review + assign to complete the process.

Use Azure CLI with SAS

This section demonstrates how to set up Azure Event Hubs authentication using shared access signatures (SAS) through Azure CLI.

The Azure CLI allows you to script and automate administrative tasks in Azure. The CLI can be installed on your local computer or run directly from the Azure Cloud Shell.

Important

Authenticating Event Hubs by using shared access signatures (SAS) will be deprecated in the future. We recommend using Microsoft Entra ID role-based access control (RBAC) instead. Follow the guidance to migrate to RBAC.

# --------------
# TODO: update the following values
#sets the name of the resource group
resourcegroup=rg-graphevents-dev
#sets the location of the resources
location='uk south'
#sets the name of the Azure Event Hubs namespace
evhamespacename=evh-graphevents-dev
#sets the name of the hub under the namespace
evhhubname=graphevents
#sets the name of the access policy to the hub
evhpolicyname=grapheventspolicy
#sets the name of the Azure KeyVault
keyvaultname=kv-graphevents
#sets the name of the secret in Azure KeyVault that will contain the connection string to the hub
keyvaultsecretname=grapheventsconnectionstring
# --------------
az group create --location $location --name $resourcegroup
az eventhubs namespace create --name $evhamespacename --resource-group $resourcegroup --sku Basic --location $location
az eventhubs eventhub create --name $evhhubname --namespace-name $evhamespacename --resource-group $resourcegroup --partition-count 2 --message-retention 1
az eventhubs eventhub authorization-rule create --name $evhpolicyname --eventhub-name $evhhubname --namespace-name $evhamespacename --resource-group $resourcegroup --rights Send
evhprimaryconnectionstring=`az eventhubs eventhub authorization-rule keys list --name $evhpolicyname --eventhub-name $evhhubname --namespace-name $evhamespacename --resource-group $resourcegroup --query "primaryConnectionString" --output tsv`
az keyvault create --name $keyvaultname --resource-group $resourcegroup --location $location --enable-soft-delete true --sku standard --retention-days 90
az keyvault secret set --name $keyvaultsecretname --value $evhprimaryconnectionstring --vault-name $keyvaultname --output none
graphspn=`az ad sp list --display-name 'Microsoft Graph Change Tracking' --query "[].appId" --output tsv`
az keyvault set-policy --name $keyvaultname --resource-group $resourcegroup --secret-permissions get --spn $graphspn --output none
keyvaulturi=`az keyvault show --name $keyvaultname --resource-group $resourcegroup --query "properties.vaultUri" --output tsv`
domainname=`az ad signed-in-user show --query 'userPrincipalName' | cut -d '@' -f 2 | sed 's/\"//'`
notificationUrl="EventHub:${keyvaulturi}secrets/${keyvaultsecretname}?tenantId=${domainname}"
echo "Notification Url:\n${notificationUrl}"

Note: The script provided here is compatible with Linux-based shells, Windows WSL, and Azure Cloud Shell. It requires some updates to run in Windows shells.

Use the Azure portal with SAS

This section demonstrates how to set up Azure Event Hubs authentication using shared access signatures (SAS) through the Azure portal.

Important

Authenticating Event Hubs by using shared access signatures (SAS) will be deprecated in the future. We recommend using Microsoft Entra ID role-based access control (RBAC) instead. Follow the guidance to migrate to RBAC.

Configure the event hub

In this section, you:

• Create an Event Hub namespace.
• Add a hub to that namespace to relay and deliver notifications.
• Add a shared access policy that allows you to get a connection string to the newly created hub.

Steps:

1. Sign in to the Azure portal with privileges to create resources in your Azure subscription.
2. Select Create a resource, type Event Hubs in the search bar, and then select the Event Hubs suggestion.
3. On the Event Hubs creation page, select Create.
4. Fill in the Event Hubs namespace creation details, and then select Create.
5. When the Event Hubs namespace is provisioned, go to the page for the namespace.
6. Select Event Hubs and then + Event Hub.
7. Give a name to the new event hub, and select Create.
8. After the event hub is created, select the name of the event hub, then choose Shared access policies and + Add to add a new policy.
9. Give a name to the policy, check Send, and select Create.
10. After the policy is created, select the name of the policy to open the details panel, and then copy the Connection string-primary key value. Record the value; you need it for the next step.

Configure the Azure Key Vault

In order to access the event hub securely and to allow for key rotations, Microsoft Graph gets the connection string to the event hub through Azure Key Vault.

In this section, you:

• Create an Azure Key Vault for storing the secret.
• Add the connection string to the event hub as a secret.
• Add an access policy for Microsoft Graph to access the secret.

Steps:

1. Sign in to the Azure portal with privileges to create resources in your Azure subscription.
2. Select Create a resource, type Key Vault in the search bar, and then select the Key Vault suggestion.
3. On the Key Vault creation page, select Create.
4. Fill in the Key Vault creation details, then select Review + Create and Create.
5. Go to the newly created key vault using the Go to resource from the notification.
6. Copy the DNS name; you need it later in this article.
7. Go to Secrets and select + Generate/Import.
8. Give a name to the secret, and keep the name for later; you need it later in this article. For the value, paste in the connection string you generated at the Event Hubs step. Select Create.
9. Select Access Policies and then + Add Access Policy.
10. For Secret permissions, select Get, and for Select Principal, select Microsoft Graph Change Tracking. Select Add.

Create the subscription and receive notifications

After you create the required Azure KeyVault and Azure Event Hubs services, you can now create your change notification subscription and start receiving change notifications via Azure Event Hubs.

Create the subscription

Creating a subscription to receive change notifications with Event Hubs is nearly identical to webhook subscription creation, but with important changes in the notificationUrl property. First review webhook subscription creation steps before continuing.

At subscription creation, the notificationUrl must point to your Event Hubs location.

Using RBAC

If you're using role-based access control, the notificationUrl property looks like this:

EventHub:https://<eventhubnamespace>.servicebus.windows.net/eventhubname/<eventhubname>?tenantId=<domainname>

• <eventhubnamespace> is the name you give to the Event Hubs namespace. It can be found on the Event Hubs Overview page under Host name.
• <eventhubname> is the name you give to the event hub. It can be found in the Event Hubs -> Overview -> Event Hubs.
• <domainname> is the name of your tenant; for example, contoso.com. Because this domain is used to access the Azure Event Hubs, it's important that it matches the domain used by the Azure subscription that holds the Azure Event Hubs. To get this information, select the Microsoft Entra ID menu on the Azure portal and check the Overview page. The domain name is displayed under the Primary domain.

Using Key Vault

If you're using Key Vault, the notificationUrl property looks like this: EventHub:https://<azurekeyvaultname>.vault.azure.net/secrets/<secretname>?tenantId=<domainname>, with the following values:

• <azurekeyvaultname> - The name you gave to the key vault during creation. It can be found in the DNS name.
• <secretname> - The name you gave to the secret during creation. It can be found on the Azure Key Vault Secrets page.
• <domainname> - The name of your tenant; for example, contoso.com. Because this domain is used to access the Azure Key Vault, it's important that it matches the domain used by the Azure subscription that holds the Azure Key Vault. To get this information, you can go to the overview page of the Azure Key Vault you created and select the subscription. The domain name is displayed under the Directory field.

Note

Duplicate subscriptions aren't allowed. When a subscription request contains the same values for changeType and resource that an existing subscription contains, the request fails with an HTTP error code 409 Conflict, and the error message Subscription Id <> already exists for the requested combination.

Migrate an event hub authentication to Microsoft Entra ID RBAC

Authenticating Event Hubs by using shared access signatures (SAS) will be deprecated in the future. We recommend authenticating Event Hubs by using Microsoft Entra ID role-based access control (RBAC) instead.

This section guides you through how to migrate your existing Event Hubs with SAS authentication to Microsoft Entra ID RBAC authentication. Use the same event hub namespace that you used with the SAS authentication, either via Azure CLI or the Azure portal.

1. Under the same event hub namespace that you're using for your existing subscription, create a new event hub.
2. Create a new subscription with the same details as the existing one, except using the new event hub's name from the preceding step in the URL. For more information, see Create the subscription: Using RBAC.

You'll receive notifications on the new event hub. You can validate if the traffic looks similar to the old subscription by inspecting the Messages chart for the event hub. Also validate for any errors or failures in receiving notifications.

After you validate that you're receiving notifications and the new event hub works correctly, you can delete the old subscription, the old event hub, and SAS-based authentication and start using the new one.

Receive notifications

Change notifications are now delivered to your application by Event Hubs. For details, see receiving events in the Event Hubs documentation.

Before you can receive the notifications in your application, you need to create another shared access policy with a "Listen" permission and obtain the connection string, similar to the steps listed in Configure the event hub.

Tip

Create a separate policy for the application that listens to Event Hubs messages instead of reusing the same connection string you set in Azure KeyVault. This separation follows the principle of least privilege by ensuring that each component of the solution has only the permissions it needs.

Handling validation notifications

Your application receives validation notifications whenever it creates a new subscription. You should ignore these notifications. The following example represents the body of a validation message.

 {
    "value":[
        {
            "subscriptionId":"NA",
            "subscriptionExpirationDateTime":"NA",
            "clientState":"NA",
            "changeType":"Validation: Testing client application reachability for subscription Request-Id: 522a8e7e-096a-494c-aaf1-ac0dcfca45b7",
            "resource":"NA",
            "resourceData":{
                "@odata.type":"NA",
                "@odata.id":"NA",
                "id":"NA"
            }
        }
    ]
}

Subscriptions for rich notifications with large payloads

The maximum message size for Event Hubs is 1 MB. When you use rich notifications, you might expect notifications that exceed this limit. To receive notifications larger than 1 MB through Event Hubs, you must also add a blob storage account to your subscription request.

Set up storage and create a subscription

1. Create a storage account.
2. Create a container in the storage account. The container name must be set to microsoft-graph-change-notifications.
3. Retrieve the storage account access keys or connection string.
4. Add the connection string to the key vault and give it a name. This value is the secret name.
5. Create or recreate your subscription, now including the blobStoreUrl property in the following syntax: blobStoreUrl: "https://<azurekeyvaultname>.vault.azure.net/secrets/<secretname>?tenantId=<domainname>"

Receive rich notifications

When Event Hubs receives a notification payload that is larger than 1 MB, the notification doesn't contain the resource, resourceData, and encryptedContent properties that are included in rich notifications. The notification instead contains an additionalPayloadStorageId property with an ID that points to the blob in your storage account where these properties are stored.

What if the Microsoft Graph Change Tracking application is missing?

The Microsoft Graph Change Tracking service principal might be missing from your tenant, depending on when the tenant was created and administrative operations. The service principal's globally unique appId is 0bf30f3b-4a52-48df-9a82-234910c4a086 and you can run the following query to confirm whether it exists in the tenant.

HTTP

GET https://graph.microsoft.com/v1.0/servicePrincipals(appId='0bf30f3b-4a52-48df-9a82-234910c4a086')

C#

// Code snippets are only available for the latest version. Current version is 5.x

// To initialize your graphClient, see https://learn.microsoft.com/en-us/graph/sdks/create-client?from=snippets&tabs=csharp
var result = await graphClient.ServicePrincipalsWithAppId("{appId}").GetAsync();

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

CLI

mgc service-principals-with-app-id get --app-id {app-id}

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

Go

// Code snippets are only available for the latest major version. Current major version is $v1.*

// Dependencies
import (
	  "context"
	  msgraphsdk "github.com/microsoftgraph/msgraph-sdk-go"
	  //other-imports
)


// To initialize your graphClient, see https://learn.microsoft.com/en-us/graph/sdks/create-client?from=snippets&tabs=go
appId := "{appId}"
servicePrincipals, err := graphClient.ServicePrincipalsWithAppId(&appId).Get(context.Background(), nil)

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

Java

// Code snippets are only available for the latest version. Current version is 6.x

GraphServiceClient graphClient = new GraphServiceClient(requestAdapter);

ServicePrincipal result = graphClient.servicePrincipalsWithAppId("{appId}").get();

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

JavaScript

Snippet not available

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

PHP

<?php
use Microsoft\Graph\GraphServiceClient;


$graphServiceClient = new GraphServiceClient($tokenRequestContext, $scopes);


$result = $graphServiceClient->servicePrincipalsWithAppId('{appId}', )->get()->wait();

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

PowerShell

Import-Module Microsoft.Graph.Applications

Get-MgServicePrincipalByAppId

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

Python

# Code snippets are only available for the latest version. Current version is 1.x
from msgraph import GraphServiceClient
# To initialize your graph_client, see https://learn.microsoft.com/en-us/graph/sdks/create-client?from=snippets&tabs=python

result = await graph_client.service_principals_with_app_id("{appId}").get()

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

If the service principal doesn't exist, create it as follows. You must grant the calling app the Application.ReadWrite.All permission to run this operation.

Method 1

HTTP

POST https://graph.microsoft.com/v1.0/servicePrincipals
Content-type: application/json

{
    "appId": "0bf30f3b-4a52-48df-9a82-234910c4a086"
}

C#

// Code snippets are only available for the latest version. Current version is 5.x

// Dependencies
using Microsoft.Graph.Models;

var requestBody = new ServicePrincipal
{
	AppId = "0bf30f3b-4a52-48df-9a82-234910c4a086",
};

// To initialize your graphClient, see https://learn.microsoft.com/en-us/graph/sdks/create-client?from=snippets&tabs=csharp
var result = await graphClient.ServicePrincipals.PostAsync(requestBody);

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

CLI

mgc service-principals create --body '{\
    "appId": "0bf30f3b-4a52-48df-9a82-234910c4a086"\
}\
'

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

Go

// Code snippets are only available for the latest major version. Current major version is $v1.*

// Dependencies
import (
	  "context"
	  msgraphsdk "github.com/microsoftgraph/msgraph-sdk-go"
	  graphmodels "github.com/microsoftgraph/msgraph-sdk-go/models"
	  //other-imports
)

requestBody := graphmodels.NewServicePrincipal()
appId := "0bf30f3b-4a52-48df-9a82-234910c4a086"
requestBody.SetAppId(&appId) 

// To initialize your graphClient, see https://learn.microsoft.com/en-us/graph/sdks/create-client?from=snippets&tabs=go
servicePrincipals, err := graphClient.ServicePrincipals().Post(context.Background(), requestBody, nil)

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

Java

// Code snippets are only available for the latest version. Current version is 6.x

GraphServiceClient graphClient = new GraphServiceClient(requestAdapter);

ServicePrincipal servicePrincipal = new ServicePrincipal();
servicePrincipal.setAppId("0bf30f3b-4a52-48df-9a82-234910c4a086");
ServicePrincipal result = graphClient.servicePrincipals().post(servicePrincipal);

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

JavaScript

const options = {
	authProvider,
};

const client = Client.init(options);

const servicePrincipal = {
    appId: '0bf30f3b-4a52-48df-9a82-234910c4a086'
};

await client.api('/servicePrincipals')
	.post(servicePrincipal);

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

PHP

<?php
use Microsoft\Graph\GraphServiceClient;
use Microsoft\Graph\Generated\Models\ServicePrincipal;


$graphServiceClient = new GraphServiceClient($tokenRequestContext, $scopes);

$requestBody = new ServicePrincipal();
$requestBody->setAppId('0bf30f3b-4a52-48df-9a82-234910c4a086');

$result = $graphServiceClient->servicePrincipals()->post($requestBody)->wait();

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

PowerShell

Import-Module Microsoft.Graph.Applications

$params = @{
	appId = "0bf30f3b-4a52-48df-9a82-234910c4a086"
}

New-MgServicePrincipal -BodyParameter $params

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

Python

# Code snippets are only available for the latest version. Current version is 1.x
from msgraph import GraphServiceClient
from msgraph.generated.models.service_principal import ServicePrincipal
# To initialize your graph_client, see https://learn.microsoft.com/en-us/graph/sdks/create-client?from=snippets&tabs=python
request_body = ServicePrincipal(
	app_id = "0bf30f3b-4a52-48df-9a82-234910c4a086",
)

result = await graph_client.service_principals.post(request_body)

Read the SDK documentation for details on how to add the SDK to your project and create an authProvider instance.

Method 2

POST https://graph.microsoft.com/v1.0/servicePrincipals(appId='0bf30f3b-4a52-48df-9a82-234910c4a086')
Content-type: application/json
Prefer: create-if-missing

{
    "displayName": "Microsoft Graph Change Tracking"
}

Related content

• Change notifications overview
• See the following Azure Event Hubs quick starts:
  • .NET Core
  • Java
  • Python
  • JavaScript