Microsoft 365 Unified Audit Logs connector reference

Important

This feature is in Beta. To use it, a workspace admin must turn on Lakeflow Connect for Microsoft 365 Unified Audit Logs from the Previews page. See Manage Azure Databricks previews.

The managed Microsoft 365 Unified Audit Logs connector supports five unified audit log source tables. This reference lists the tables, destination table schemas, connection options, subscription plans, and Microsoft 365 permissions.

Supported source tables

The Microsoft 365 Unified Audit Logs connector supports the following source tables in the default source schema. All five tables use lw_id as the primary key, use incremental sync, and use time as the cursor field.

Source table Primary key Description Microsoft Management Activity API content type Sync mode Cursor field
audit_azure_active_directory lw_id Microsoft Entra ID events, including sign-ins, application access, and directory changes. Audit.AzureActiveDirectory Incremental time
audit_exchange lw_id Exchange events, including mailbox, folder, and item operations. Audit.Exchange Incremental time
audit_sharepoint lw_id SharePoint and OneDrive events, including file, folder, sharing, and site operations. Audit.SharePoint Incremental time
audit_general lw_id Microsoft 365 audit events that are not included in the other workload-specific content types. Audit.General Incremental time
dlp_all lw_id Data loss prevention events across supported Microsoft 365 workloads. DLP.All Incremental time

For field definitions and workload-specific event schemas, see the Microsoft 365 Management Activity API schema. Microsoft can add fields and event types to these source payloads.

Use the following source names in a pipeline definition:

objects:
  - table:
      source_schema: 'default'
      source_table: 'audit_azure_active_directory'
  - table:
      source_schema: 'default'
      source_table: 'audit_exchange'
  - table:
      source_schema: 'default'
      source_table: 'audit_sharepoint'
  - table:
      source_schema: 'default'
      source_table: 'audit_general'
  - table:
      source_schema: 'default'
      source_table: 'dlp_all'

For a complete pipeline definition, see Examples.

Destination table schemas

All tables use lw_id as the primary key and time as the cursor field. Each destination table also includes the source fields that Microsoft returns for its content type. Microsoft can add fields and event types to these source payloads, so for the complete, authoritative field list, see the Microsoft 365 Management Activity API schema.

audit_azure_active_directory

Primary key: lw_id Cursor field: time

Field Data type
lw_id STRING
time TIMESTAMP
Id STRING
RecordType INT
CreationTime TIMESTAMP
Operation STRING
OrganizationId STRING
UserType INT
UserKey STRING
Workload STRING
Version INT
ResultStatus STRING
ObjectId STRING
UserId STRING
ClientIP STRING
Scope STRING
AppAccessContext STRUCT<AADSessionId: STRING, APIId: STRING, ClientAppId: STRING, ClientAppName: STRING, CorrelationId: STRING>
AzureActiveDirectoryEventType INT
ExtendedProperties ARRAY<STRUCT<Name: STRING, Value: STRING>>
ModifiedProperties ARRAY<MAP<STRING, VARIANT>>
DeviceProperties ARRAY<STRUCT<Name: STRING, Value: STRING>>
Application STRING
Client STRING
LoginStatus INT
UserDomain STRING
Actor ARRAY<STRUCT<ID: STRING, Type: INT>>
ActorContextId STRING
ActorIpAddress STRING
InterSystemsId STRING
IntraSystemId STRING
SupportTicketId STRING
Target ARRAY<STRUCT<ID: STRING, Type: INT>>
TargetContextId STRING
ApplicationId STRING
ErrorNumber STRING
LogonError STRING

audit_exchange

Primary key: lw_id Cursor field: time

Field Data type
lw_id STRING
time TIMESTAMP
Id STRING
RecordType INT
CreationTime TIMESTAMP
Operation STRING
OrganizationId STRING
UserType INT
UserKey STRING
Workload STRING
Version INT
ResultStatus STRING
ObjectId STRING
UserId STRING
ClientIP STRING
Scope STRING
AppAccessContext STRUCT<AADSessionId: STRING, APIId: STRING, ClientAppId: STRING, ClientAppName: STRING, CorrelationId: STRING>
ModifiedObjectResolvedName STRING
Parameters ARRAY<STRUCT<Name: STRING, Value: STRING>>
ModifiedProperties ARRAY<MAP<STRING, VARIANT>>
ExternalAccess BOOLEAN
OriginatingServer STRING
OrganizationName STRING
LogonType INT
InternalLogonType INT
MailboxGuid STRING
MailboxOwnerUPN STRING
MailboxOwnerSid STRING
MailboxOwnerMasterAccountSid STRING
LogonUserSid STRING
LogonUserDisplayName STRING
ClientInfoString STRING
ClientIPAddress STRING
ClientMachineName STRING
ClientProcessName STRING
ClientVersion STRING
Folder STRUCT<Id: STRING, Path: STRING, FolderItems: ARRAY<STRUCT<Id: STRING, ImmutableId: STRING, InternetMessageId: STRING, CreationTime: TIMESTAMP, Subject: STRING, SizeInBytes: BIGINT, Sensitivity: STRING, ClientRequestId: STRING, Teams: VARIANT>>>
CrossMailboxOperations BOOLEAN
DestMailboxId STRING
DestMailboxOwnerUPN STRING
DestMailboxOwnerSid STRING
DestMailboxOwnerMasterAccountSid STRING
DestFolder STRUCT<Id: STRING, Path: STRING, FolderItems: ARRAY<STRUCT<Id: STRING, ImmutableId: STRING, InternetMessageId: STRING, CreationTime: TIMESTAMP, Subject: STRING, SizeInBytes: BIGINT, Sensitivity: STRING, ClientRequestId: STRING, Teams: VARIANT>>>
Folders ARRAY<STRUCT<Id: STRING, Path: STRING, FolderItems: ARRAY<STRUCT<Id: STRING, ImmutableId: STRING, InternetMessageId: STRING, CreationTime: TIMESTAMP, Subject: STRING, SizeInBytes: BIGINT, Sensitivity: STRING, ClientRequestId: STRING, Teams: VARIANT>>>>
AffectedItems ARRAY<STRUCT<Id: STRING, Subject: STRING, ParentFolder: STRUCT<Id: STRING, Path: STRING, FolderItems: ARRAY<STRUCT<Id: STRING, ImmutableId: STRING, InternetMessageId: STRING, CreationTime: TIMESTAMP, Subject: STRING, SizeInBytes: BIGINT, Sensitivity: STRING, ClientRequestId: STRING, Teams: VARIANT>>>, Attachments: STRING>>
Item STRUCT<Id: STRING, Subject: STRING, ParentFolder: STRUCT<Id: STRING, Path: STRING, FolderItems: ARRAY<STRUCT<Id: STRING, ImmutableId: STRING, InternetMessageId: STRING, CreationTime: TIMESTAMP, Subject: STRING, SizeInBytes: BIGINT, Sensitivity: STRING, ClientRequestId: STRING, Teams: VARIANT>>>, Attachments: STRING>
SendAsUserSmtp STRING
SendAsUserMailboxGuid STRING
SendOnBehalfOfUserSmtp STRING
SendOnBehalfOfUserMailboxGuid STRING

audit_sharepoint

Primary key: lw_id Cursor field: time

Field Data type
lw_id STRING
time TIMESTAMP
Id STRING
RecordType INT
CreationTime TIMESTAMP
Operation STRING
OrganizationId STRING
UserType INT
UserKey STRING
Workload STRING
Version INT
ResultStatus STRING
ObjectId STRING
UserId STRING
ClientIP STRING
Scope STRING
AppAccessContext STRUCT<AADSessionId: STRING, APIId: STRING, ClientAppId: STRING, ClientAppName: STRING, CorrelationId: STRING>
Site STRING
ItemType STRING
EventSource STRING
SourceName STRING
UserAgent STRING
MachineDomainInfo STRING
MachineId STRING
ListItemUniqueId STRING
ListId STRING
ApplicationId STRING
ApplicationDisplayName STRING
IsWorkflow BOOLEAN
SiteUrl STRING
SourceRelativeUrl STRING
SourceFileName STRING
SourceFileExtension STRING
DestinationRelativeUrl STRING
DestinationFileName STRING
DestinationFileExtension STRING
UserSharedWith STRING
SharingType STRING
SourceLabel STRING
DestinationLabel STRING
SensitivityLabelOwnerEmail STRING
SensitivityLabelId STRING
ListTitle STRING
ListName STRING
ListUrl STRING
ListBaseType STRING
ListBaseTemplateType STRING
IsHiddenList BOOLEAN
IsDocLib BOOLEAN
TargetUserOrGroupName STRING
TargetUserOrGroupType STRING
UniqueSharingId STRING
CustomEvent STRING
EventData STRING
ModifiedProperties ARRAY<MAP<STRING, VARIANT>>

audit_general

Primary key: lw_id Cursor field: time

Field Data type
lw_id STRING
time TIMESTAMP
Id STRING
RecordType INT
CreationTime TIMESTAMP
Operation STRING
OrganizationId STRING
UserType INT
UserKey STRING
Workload STRING
Version INT
ResultStatus STRING
ObjectId STRING
UserId STRING
ClientIP STRING
Scope STRING
AppAccessContext STRUCT<AADSessionId: STRING, APIId: STRING, ClientAppId: STRING, ClientAppName: STRING, CorrelationId: STRING>
payload VARIANT

dlp_all

Primary key: lw_id Cursor field: time

Field Data type
lw_id STRING
time TIMESTAMP
Id STRING
RecordType INT
CreationTime TIMESTAMP
Operation STRING
OrganizationId STRING
UserType INT
UserKey STRING
Workload STRING
Version INT
ResultStatus STRING
ObjectId STRING
UserId STRING
ClientIP STRING
Scope STRING
AppAccessContext STRUCT<AADSessionId: STRING, APIId: STRING, ClientAppId: STRING, ClientAppName: STRING, CorrelationId: STRING>
SharePointMetaData STRUCT<From: STRING, itemCreationTime: TIMESTAMP, SiteCollectionGuid: STRING, SiteCollectionUrl: STRING, FileName: STRING, FileOwner: STRING, FilePathUrl: STRING, DocumentLastModifier: STRING, DocumentSharer: STRING, UniqueId: STRING, LastModifiedTime: TIMESTAMP, IsViewableByExternalUsers: BOOLEAN>
ExchangeMetaData STRUCT<MessageID: STRING, From: STRING, To: ARRAY<STRING>, CC: ARRAY<STRING>, BCC: ARRAY<STRING>, Subject: STRING, Sent: TIMESTAMP, RecipientCount: INT>
EndPointMetaData STRUCT<SensitiveInformation: ARRAY<STRUCT<Confidence: INT, Count: INT, Location: STRING, SensitiveType: STRING, SensitiveInformationDetections: STRUCT<DetectedValues: ARRAY<STRUCT<Name: STRING, Value: STRING>>, ResultsTruncated: BOOLEAN>, SensitiveInformationDetailedClassificationAttributes: ARRAY<STRUCT<Confidence: INT, Count: INT, IsMatch: BOOLEAN>>, SensitiveInformationTypeName: STRING, UniqueCount: INT>>, EnforcementMode: STRING, FileExtension: STRING, FileType: STRING, DeviceName: STRING>
ExceptionInfo STRING
PolicyDetails ARRAY<STRUCT<PolicyId: STRING, PolicyName: STRING, Rules: ARRAY<STRUCT<RuleId: STRING, RuleName: STRING, Actions: ARRAY<STRING>, OverriddenActions: ARRAY<STRING>, Severity: STRING, RuleMode: STRING, ConditionsMatched: STRUCT<SensitiveInformation: ARRAY<STRUCT<Confidence: INT, Count: INT, Location: STRING, SensitiveType: STRING, SensitiveInformationDetections: STRUCT<DetectedValues: ARRAY<STRUCT<Name: STRING, Value: STRING>>, ResultsTruncated: BOOLEAN>, SensitiveInformationDetailedClassificationAttributes: ARRAY<STRUCT<Confidence: INT, Count: INT, IsMatch: BOOLEAN>>, SensitiveInformationTypeName: STRING, UniqueCount: INT>>, DocumentProperties: ARRAY<STRUCT<Name: STRING, Value: STRING>>, OtherConditions: ARRAY<STRUCT<Name: STRING, Value: STRING>>>>>>>
SensitiveInfoDetectionIsIncluded BOOLEAN

Connection options

Option Type Required Description
tenant_id String Yes The Directory (tenant) ID of the Microsoft Entra tenant.
client_id String Yes The Application (client) ID of the registered Microsoft Entra application.
client_secret String Yes The client secret value for the registered application.
subscription_plan String Yes The Microsoft cloud environment for the tenant. Catalog Explorer initially selects enterprise.

Subscription plans

Display name Value Management Activity API host
Enterprise enterprise manage.office.com
Government GCC gcc manage-gcc.office.com
Government GCC High gcc_high manage.office365.us
Government DoD dod manage.protection.apps.mil

Required Microsoft 365 permissions

Permission Type Required for
ActivityFeed.Read Application Reading Audit.AzureActiveDirectory, Audit.Exchange, Audit.SharePoint, and Audit.General content.
ActivityFeed.ReadDlp Application Reading DLP.All content for the dlp_all source table.

An administrator must grant tenant-wide consent for these permissions. The connector doesn't use delegated permissions.

  • Last updated on