Microsoft Entra ID audit logs

Important

Lakewatch is in Private Preview. To request access, reach out to your Azure Databricks account team.

Overview

Microsoft Entra ID audit logs capture directory and administrative changes, such as changes to users, groups, applications, roles, and policies. Password policy changes are recorded in the same log. Lakewatch ingests the AuditLogs diagnostic category from cloud storage using Auto Loader and schematizes the data with the built-in azure_audit parser. This is a cloud storage source, not an API connector.

Sign-in, Microsoft Graph activity, Identity Protection, and provisioning logs have their own parsers. See Supported logs.

Onboard Microsoft Entra ID audit logs

To ingest Microsoft Entra ID audit logs, create a cloud-storage datasource for them and attach the built-in azure_audit parser so events are written to a schematized parsed table. For prerequisites, Unity Catalog access, and the full datasource setup steps, see Cloud storage.

The parser claims only events whose category is AuditLogs. If one storage location mixes several Azure log categories, attach every matching Azure parser to the same datasource, with the generic azure_monitor_activity parser last. See Ingest several Azure log categories from one location.

Upgrade from version 1

Version 1 of the azure_audit parser matched every Microsoft Entra ID diagnostic category, including sign-in, Microsoft Graph activity, and Identity Protection events. Starting with version 2, it matches only AuditLogs events. Datasources keep the parser version they reference, so existing datasources are unaffected until you update them. Before you move a datasource that receives other categories to version 2, attach the matching parsers to it, or those events are routed to the dead-letter sentinel.

Parsed schema and observables

The built-in azure_audit parser schematizes Microsoft Entra ID audit logs into a parsed table. Every parsed table also carries the Lakewatch-managed columns:

• lw_id: A non-null string Lakewatch uses as the primary key, supporting deduplication across incremental reads.
• time: A non-null event timestamp assigned by the parser.
• lw_meta: Standard Lakewatch processing metadata.
• lw_observables: Values extracted from fields marked as security observables in the parser.

The full event payload is kept in the properties column as a VARIANT, so fields that the parser doesn't promote to their own columns remain queryable. For more information, see the Parser reference.

Parsed table schema

The azure_audit parser writes the following columns (generated from the parser definition):

Column	Data type
time	TIMESTAMP
resourceId	STRING
operationName	STRING
operationVersion	STRING
category	STRING
tenantId	STRING
resultSignature	STRING
resultDescription	STRING
durationMs	DOUBLE
callerIpAddress	STRING
correlationId	STRING
identity	STRING
level	STRING
auditCategory	STRING
activityDisplayName	STRING
operationType	STRING
result	STRING
resultReason	STRING
loggedByService	STRING
userAgent	STRING
initiatedByAppId	STRING
initiatedByAppDisplayName	STRING
initiatedByServicePrincipalId	STRING
initiatedByUserId	STRING
initiatedByUserDisplayName	STRING
initiatedByUserPrincipalName	STRING
initiatedByUserIp	STRING
properties	VARIANT

Parser observables

The parser extracts the following fields into lw_observables (generated from the parser definition):

Field	Observable
callerIpAddress	ip
correlationId	trace_id
initiatedByAppId	actor_id
initiatedByServicePrincipalId	actor_id
initiatedByUserId	actor_id
initiatedByUserDisplayName	username
initiatedByUserPrincipalName	username
initiatedByUserPrincipalName	email
initiatedByUserIp	ip

The following VARIANT fields are scanned for observables by matching nested key names; the observable types extracted from each:

Field	Observable types scanned
properties	username, email