Azure Content Understanding

ContentUnderstandingContextProvider analyzes file attachments with Azure Content Understanding and injects structured results into the agent context. It supports documents, images, audio, and video, including OCR, tables, structured fields, transcription, diarization, and segment summaries.

This integration uses the pre-processing pattern: it transforms incoming content before model invocation and can retain processed state for later turns.

For large documents, the provider can upload extracted markdown to a file-search vector store instead of placing the entire result in the model context.

Prerequisites

  • An Azure subscription.
  • Azure Content Understanding in a supported region.
  • The service's required model deployments.
  • Azure identity access to the resource.

Install the package

pip install agent-framework-azure-contentunderstanding --pre

Analyze a document

Attach ContentUnderstandingContextProvider to the agent and send a supported binary attachment. The provider removes the binary input after analysis and supplies the extracted content to the model.

Processing options

  • Leave analyzer_id unset to select a document, audio, or video search analyzer from the media type.
  • Set max_wait=None when the run must wait for analysis to complete.
  • Use FileSearchConfig for token-efficient retrieval over large extracted documents. Scope the vector store and matching file_search tool to the intended user, tenant, or sharing group.
  • Reuse an AgentSession to preserve analyzed-document state across turns. A session doesn't isolate retrieval from a shared vector store.

Next steps

Go deeper: