Best approach to ingest Entra ID Identity Protection and Audit events — Graph API vs Event Hub

Question

Hi Team,

We are building an ingestion connector for security monitoring and want to ingest specific Microsoft Entra ID events related to identity risk and audit activities.

Use Case 1: Pull Risky Sign-Ins

We want to receive risky sign-in alerts from Entra ID.

Relevant endpoint:

• /identityProtection/riskySignIns

---

Use Case 2: Pull Identity Risk Detection

Entra ID generates alerts when suspicious identity activity or potential account risk is detected. We want to automatically collect these alerts, evaluate severity, correlate them with other security signals, and enable faster response actions.

Relevant endpoints:

/identityProtection/riskDetections

/identityProtection/riskyUsers

---

Use Case 3: Pull Audit Logs

Entra ID audit logs capture directory activities such as user and group management, application registrations, role assignments, Conditional Access policy changes, and other administrative actions.

Relevant endpoint:

/auditLogs/directoryAudits

---

Our Requirement

To ingest these events, we initially explored the Microsoft Graph API, using the endpoints listed above. However, in our earlier experience with some Microsoft Graph APIs (for example, the alerts API), we observed latency issues where events were sometimes delayed by 1–48 hours. Because of this, we are unsure whether similar latency might occur with the Identity Protection endpoints mentioned above. Plese correct us if this is not the case in above endpoints, then we will procees with these endpoints only.

We are now evaluating whether using Event Hub streaming for Entra logs would be a better approach.

Our requirements are:

1. We want to ingest only the above event types, without polling unnecessary data.
2. We want near real-time ingestion.
3. For the initial run, we need to retrieve historical data for the past 14 days (historical polling/backfill).

---

Clarifications Needed

For these use cases, is it recommended to use Microsoft Graph API polling or Event Hub streaming?

1. Can Event Hub export provide these Identity Protection events (riskySignIns, riskDetections, riskyUsers) or is it limited to certain log categories?
2. If we use Event Hub, how can we perform the initial 14-day historical ingestion?
3. Is there a recommended architecture that avoids unnecessary polling while still ensuring we only ingest the relevant events?
4. Any guidance or best practices would be greatly appreciated.

Thanks!

Answer 1

Hey Nisha, it looks like you want near real-time ingestion of identity-risk alerts (riskySignIns, riskDetections, riskyUsers) plus directory audit events—without over-polling—and you need a 14-day backfill on day one. Here’s what I’d recommend:

1. Use a hybrid pipeline • Identity Protection events (riskySignIns, riskDetections, riskyUsers) are only exposed via Microsoft Graph. There’s no direct Event Hub export for those. • Directory audit logs (and Sign-In logs) can be routed via Diagnostic Settings → Event Hub for continuous streaming.
2. Historical backfill (first 14 days) • For Identity Protection endpoints and auditLogs/directoryAudits: call the Graph API /identityProtection/... and /auditLogs/directoryAudits with a createdDateTime ge {T-14d} filter to pull the last 14 days in a single or paged run. • Store the highest createdDateTime or lastUpdatedDateTime you see, then switch to incremental.
3. Ongoing real-time ingestion • Audit & Sign-In logs: let Event Hub deliver them as they occur. In Diagnostic Settings, select only “AuditLogs” (and “SignInLogs” if you want raw sign-ins) so you don’t stream extra data. Filter further downstream if you only care about risky sign-ins. • Identity Protection: schedule a short-interval Graph poll (e.g. every 5 – 15 minutes) against /identityProtection/riskySignIns, /identityProtection/riskDetections, /identityProtection/riskyUsers using a filter on your last checkpoint timestamp.
4. Avoid unnecessary polling • In Diagnostic Settings you choose exactly which log categories go to Event Hub. • In your Graph poller, keep state (lastProcessedTime) so each run only requests new records via a createdDateTime or lastUpdatedDateTime filter.

Putting it all together:

• Day 1: backfill both auditLogs and identityProtection events via Graph (14 days)

• Ongoing:

– AuditLogs/SignInLogs via Event Hub → your consumer → SIEM or data lake

– Identity Protection via scheduled Graph polling with time filters

This gives you near real-time, no surplus data, and a one-time historical pull.

— Reference list —

1. Stream Entra activity logs to Event Hub: https://learn.microsoft.com/entra/identity/monitoring-health/howto-stream-logs-to-event-hub
2. Audit log activity details & latency: https://learn.microsoft.com/entra/identity/monitoring-health/reference-audit-activities https://learn.microsoft.com/entra/identity/monitoring-health/reference-log-latency
3. Graph Identity Protection endpoints: https://learn.microsoft.com/graph/api/resources/identityprotection-root (see /identityProtection/riskySignIns, /riskDetections, /riskyUsers)
4. Graph directory audit logs: https://learn.microsoft.com/graph/api/directoryaudit-list
5. Diagnostic Settings for routing logs: https://learn.microsoft.com/azure/active-directory/reports-monitoring/concept-activity-logs-azure-monitor

Note: This content was drafted with the help of an AI system. Please verify the information before relying on it for decision-making.

Answer 2

For the three scenarios, Microsoft Graph is the primary and recommended interface; Event Hubs is complementary and does not fully replace the Identity Protection APIs.

1. Identity Protection data (risky sign-ins, risk detections, risky users)

• Identity Protection risk data is exposed through dedicated Microsoft Graph APIs:
  • riskDetections – list of user and sign-in linked risk detections.
  • riskyUsers – list of users detected as risky, plus APIs to confirm compromised or dismiss.
  • signIn – sign-ins with risk-related properties (risk state, detail, level).
• These APIs are explicitly called out as the way to query user and service principal risks detected by Microsoft Entra ID Protection.
• Identity Protection is a premium feature; access to full reports and data via these APIs requires the appropriate Microsoft Entra ID Protection licenses.
• Data availability is governed by Microsoft Entra data retention policies.

From the provided material, Identity Protection risk detections and risky users are designed to be accessed via Microsoft Graph, not via Event Hub export. Event Hub export is documented as a way to stream “activity logs” (audit and sign-in logs), not as a replacement for the Identity Protection resource types.

Implication:

• For riskySignIns (via sign-in logs with risk properties), riskDetections, and riskyUsers, use Microsoft Graph APIs.
• Event Hub streaming cannot be assumed to provide the same structured Identity Protection entities (riskDetections, riskyUsers) as the Graph Identity Protection APIs.

2. Audit logs and sign-ins

• Microsoft Entra activity logs (including audit and sign-in logs) can be accessed in several ways:
  1. Stream activity logs to an Event Hub.
  2. Access activity logs through the Microsoft Graph API.
  3. Integrate with Azure Monitor logs.
  4. Monitor in real time with Microsoft Sentinel.
  5. View in the Entra admin center.
  6. Export for storage and queries.
• The FAQ explicitly states that the newer Microsoft Graph endpoint provides two key reports for what used to be obtained from the legacy graph.windows.net reports endpoint:
  • Audit
  • Sign-ins
• These reports provide all the data previously available from the old API and can be used to get app usage, device usage, and user sign-in information.

Implication:

• For /auditLogs/directoryAudits and sign-in logs, either Microsoft Graph or Event Hub streaming is valid.
• Event Hub is well-suited for high-volume, near real-time streaming of audit and sign-in events into a SIEM or custom pipeline.

3. Latency and ingestion pattern

The context does not provide explicit latency SLAs for Identity Protection endpoints vs Event Hub, but it does describe:

• Event Hubs:
  • High availability and low latency.
  • Ability to process millions of events per second.
  • Retention up to 90 days depending on tier, allowing consumers to process at their own pace.
• A recommended pattern for high-volume, near real-time security analysis is a Pub/Sub pattern using Microsoft Graph change notifications delivered to Azure Event Hubs, with downstream processing via Azure Functions or App Services.

Given this:

• For near real-time streaming of audit and sign-in logs, Event Hub export is recommended.
• For Identity Protection entities (riskDetections, riskyUsers), Microsoft Graph is the designed access method; use direct Graph polling or Graph change-notification patterns where available.

4. Historical backfill (14 days)

• Identity Protection data availability is governed by Microsoft Entra data retention policies.
• For an initial 14-day backfill:
  • Use Microsoft Graph to query the relevant resources with date filters within the retention window.
  • For audit and sign-in logs, use the Microsoft Graph activity log APIs to pull historical data for the desired period.
  • For Identity Protection data (riskDetections, riskyUsers, sign-ins with risk properties), use the corresponding Graph resources and filter by time where supported.
• Event Hubs provide retention up to 90 days, but only from the time streaming is enabled. Event Hubs cannot backfill data from before the export was configured. Historical backfill must therefore be done via Microsoft Graph or another log store (for example, Azure Monitor logs) that already contains the historical data.

5. Recommended architecture for the described requirements

Based on the context:

• Identity Protection (risky sign-ins, risk detections, risky users):
  • Use Microsoft Graph Identity Protection APIs as the primary source.
  • Implement scheduled polling with appropriate filters (for example, by time or risk state) to minimize unnecessary data retrieval.
  • Use the Microsoft Graph PowerShell SDK or direct REST calls as appropriate.
• Audit logs and sign-ins:
  • Configure streaming of Entra activity logs to Azure Event Hubs for near real-time ingestion.
  • Use a function app or similar compute to process events from Event Hubs and forward only relevant events to the security monitoring platform.
• Initial 14-day backfill:
  • Use Microsoft Graph activity log APIs for audit and sign-in logs.
  • Use Microsoft Graph Identity Protection APIs for riskDetections, riskyUsers, and sign-ins with risk properties, constrained to the last 14 days and within the documented retention limits.
• Avoiding unnecessary polling:
  • For high-volume logs (audit/sign-ins), rely on Event Hub streaming rather than Graph polling.
  • For Identity Protection entities, design polling to be incremental (for example, by last seen timestamp or risk state) and scoped to only the needed properties.
  • Where applicable, consider Graph change-notification patterns (Pub/Sub) to push changes into Event Hubs and then process them asynchronously.

This hybrid approach aligns with the documented guidance: use Microsoft Graph for Identity Protection risk data and activity logs, and use Event Hubs as the scalable, low-latency transport for activity logs and change notifications, with historical backfill handled via Microsoft Graph within the retention window.

---

References:

• Overview of identity protection APIs in Microsoft Graph
• Microsoft Entra ID Protection and the Microsoft Graph PowerShell SDK
• How to access activity logs in Microsoft Entra ID
• Frequently asked questions around Microsoft Entra monitoring and health
• Security operations for privileged accounts in Microsoft Entra ID
• Retrieve events at your speed by using Microsoft Graph