Agent 365 observability concepts

This article explains the data model behind Agent 365 observability - what telemetry agents emit, who can emit it, where it lands, and the limits that apply. These concepts apply to every integration path: the Microsoft OpenTelemetry Distro, the Agent 365 SDK, and direct OTel.

Note

Wire-level details - the URL routes in Authentication, the HTTP error codes in Limits and drop conditions, and the per-request size and rate limits - apply specifically to the direct OTel path. The SDK and Distro abstract these for you. The rest of this article (glossary, data flow, identity models, scopes, drop conditions, where data shows up) applies to every path.

Pick your integration path

Three paths emit the same span data model into Agent 365. Pick one:

  • Microsoft OpenTelemetry Distro - recommended for new integrations. Unified observability SDK across Agent 365, Microsoft Foundry, Azure Monitor, and more.
  • Agent 365 SDK (Observability SDK) - the earlier SDK. Continues to work without breaking changes, but no longer the recommended path for new integrations; migration guidance for existing SDK users is coming.
  • Direct OTel - the raw OTLP/HTTP path. Use it only if you already have an OpenTelemetry pipeline in place, your agent framework can't use the Agent 365 SDK, or your agent is in a language the SDK doesn't yet support (such as Java).

Whichever path you pick, the data model, identity models, scopes, limits, and downstream surfaces described below all apply.

Glossary

  • App id (appId): The application identifier issued when a Microsoft Entra app or Microsoft Entra Agent ID agent identity is registered.
    • Equal to the OAuth client_id, not the Microsoft Entra object ID.
    • Throughout these docs, "agent id" and "blueprint id" both mean an appId.
  • Conversation: A logical thread of agent interactions, such as a Teams chat thread.
    • Identified by gen_ai.conversation.id.
    • The primary join key for a run.
  • Channel: The surface the agent runs in: msteams, outlook, web, and so on.
  • Run: One user message in, one agent reply out. Modeled as a tree of OTel spans sharing a traceId.

How it works

For an overview of Agent 365 and what telemetry feeds into, see Overview of Microsoft Agent 365.

You send telemetry as OpenTelemetry trace data:

  • A tree of spans describing one run (one user message in, one agent reply out).
  • Each span describes a single step - the top-level agent invocation, an LLM call, a tool call, or the final reply.

Data flow

   Your agent code

        |
        v

   +---------------+
   | OTel SDK or   |
   | raw HTTP      |
   +---------------+

        |
        v

   POST /traces  agent365.svc.cloud.microsoft

        |
        v

  +-------------------------------------+
  | Microsoft Defender                  |
  |   (CloudAppEvents table             |
  |    in advanced hunting)             |
  |                                     |
  | Microsoft Purview                   |
  |                                     |
  | Microsoft 365 admin center          |
  |   (agent inventory and              |
  |    security views)                  |
  +-------------------------------------+

Identity models

For a full explanation of agent identity models (standard Microsoft Entra app registration vs. Microsoft Entra Agent ID agent identity blueprint, including AI teammates), see Get started with Agent 365 development. Your choice of identity model determines which authentication flow and endpoint you use.

If your agent has no Microsoft Entra registration, it can't use these routes directly. Identify the agent via the alternate ID attributes (see Attribute reference) and contact the Agent 365 team about the appropriate ingress path.

Authentication

Authentication branches on whether your service authenticates itself or on behalf of a user. The branch determines the OAuth flow, the token claim that carries the permission, and the URL route.

  • Service authenticates itself: No signed-in user - autonomous, scheduled, or event-driven.

    • OAuth flow: Service-to-service (S2S) client credentials.
    • Token claim: roles.
    • URL route: /observabilityService/....

  • Service authenticates on behalf of a user: For AI teammates, or for the agent's own user account.

    • OAuth flow: On-behalf-of (OBO).
    • Token claim: scp.
    • URL route: /observability/....

The same agent app can participate in both flows, such as an AI teammate that also runs a nightly autonomous summarization pass. For more information, see autonomous app OAuth flow and on-behalf-of flow.

For the full token recipes for each combination of identity model and flow, see Authentication recipes in the Integration guide.

Agent identity is bound to the URL

The {agentId} in the URL must equal the calling application's appId (the appid or azp claim in your token). Mismatches return 403 Forbidden. For blueprint-derived identities, {agentId} is the agent identity appId, not the blueprint appId.

In addition, every span you send must set gen_ai.agent.id to the same appId; the server validates the in-payload agent identity against the authenticated agent and rejects mismatches. This step catches accidentally mixing spans from multiple agents into one request.

A scope (delegated) or app role (application) is the named permission Microsoft Entra mints into the access token. For Agent 365 telemetry, the permission is Agent365.Observability.OtelWrite on the Agent 365 Observability resource (audience 9b975845-388f-4429-889e-eab1ef63949c).

The same permission name is registered as both kinds:

  • App role for the autonomous (S2S / client-credentials) flow. Lands in the roles claim. Selected by <resource>/.default.
  • Delegated scope for the OBO flow. Lands in the scp claim. Selected by <resource>/Agent365.Observability.OtelWrite (or <resource>/.default).

Agent 365 also exposes a read-side permission, Agent365.Observability.OtelRead, used by operators who query Agent 365 telemetry. Most partners don't need it - these docs cover ingestion only.

Adding the permission to your app

  • For a standard Microsoft Entra app registration: in the Azure portal, add Agent365.Observability.OtelWrite (app role for S2S, scope for delegated) under API permissions on the agent's app registration.
  • For a blueprint: agents minted from a Microsoft Entra Agent ID agent identity blueprint inherit the OAuth permissions defined on the blueprint, so a tenant admin pre-provisions permissions once. Every agent instance built from that blueprint receives them automatically. See Configure inheritable permissions for agent identity blueprints.

Before tokens carry the role / scope, a tenant admin in the customer's tenant must grant consent. See Grant agents access to Microsoft 365 resources.

Without consent, token acquisition fails with AADSTS65001 ("user or administrator hasn't consented") or the token is issued without the roles / scp claim and the ingestion endpoint rejects the request with 403.

Consent is granted once per tenant, and applies to every instance built from a blueprint thereafter. Reconsent is only needed when a new permission is added to the blueprint.

Limits and drop conditions

Knowing these limits up front prevents surprises during integration - most are silent (the API accepts the request but data never appears downstream).

Wire-level limits:

  • api-version=1 is required on every request.
  • Maximum request body size is 1 MB. Larger requests get 413 Payload Too Large.
  • The two routes have separate rate limits. On 429, honor Retry-After (set to 1 second) and back off with jitter.

Error responses:

  • 403 Forbidden--token missing the required app role / scope, or {agentId} in the URL doesn't match the appid / azp of your token.
  • 413 Payload Too Large--body exceeds 1 MB.
  • 429 Too Many Requests--rate limit hit; honor Retry-After: 1 and back off with jitter.

Drop conditions (request accepted by HTTP but data doesn't appear downstream):

# Condition Behavior
1 Span gen_ai.operation.name missing or not in {invoke_agent, execute_tool, chat, output_messages} Per-span drop. Surfaced in partialSuccess.rejectedSpans + errorMessage.
2 No user in the customer tenant has a Microsoft 365 E7 or Microsoft Agent 365 license assigned. At least one user in the tenant must have the license assigned (the SKU being present in the tenant isn't sufficient - assignment kicks off the Defender backend workflow). The licensed user doesn't have to be the human caller of the agent. Whole request silently dropped. Returns 200 { "partialSuccess": null }.

A 200 OK is not proof of ingestion. Use the verification flow to confirm data lands.

Where your data shows up

Once accepted, your spans surface in three customer-facing experiences. All three depend on a valid invoke_agent span at the root of the run. A run with only chat / execute_tool / output_messages spans is queryable in Defender advanced hunting (the CloudAppEvents table) but is invisible to every other surface below.

Microsoft Defender. Agent activity (invoke_agent, execute_tool, chat) appears in the agent-activity views. Tenant administrators and security analysts can drill into individual runs, tools, and inference calls. The agent-activity views key off the invoke_agent span; without one, the run doesn't appear there even though child spans are still queryable via advanced hunting. The advanced-hunting view - CloudAppEvents - accepts every operation: ActionType reflects the operation (InvokeAgent, InferenceCall, ExecuteToolBySDK, ExecuteToolByGateway, ExecuteToolByMCPServer) and the per-span fields are inside RawEventData. The customer-visible field names map directly to the span attributes you sent: ConversationId ← gen_ai.conversation.id, SessionIdentity ← microsoft.session.id, AgentId ← gen_ai.agent.id, PlatformTargetAgentId ← microsoft.a365.agent.platform.id, and so on. See Attribute reference for the full mapping.

Microsoft 365 admin center. Agent activity also surfaces in the agent-inventory and security views used by tenant admins to govern agents in their tenant. The admin center ingests invoke_agent rows only: agents with no invoke_agent telemetry don't appear in the inventory, and runs that emit only chat / execute_tool / output_messages are invisible here. The attributes the admin center reads (agent id, agent name, blueprint id, caller identity, conversation id, channel, error status) all come from the invoke_agent span.

Microsoft Purview. Agent activity is also surfaced to compliance administrators in Microsoft Purview, where they can configure data-handling and policy rules over agent runs (data loss prevention, retention, communication compliance, and similar). The attributes Purview policies key off (agent id / blueprint id, caller identity, conversation / channel, request and response messages) all come from the invoke_agent span and its descendants.

Next steps

  • Attribute reference - Per-attribute specification, requirements, and value-picking guidance.
  • Troubleshooting - Verifying ingestion, common pitfalls, and error responses.
  • Last updated on