What is Azure AI Content Understanding (preview)?

ข้อสำคัญ

• Azure AI Content Understanding is available in preview. Public preview releases provide early access to features that are in active development.
• Features, approaches, and processes can change or have limited capabilities, before General Availability (GA).
• For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

Azure AI Content Understanding is a new Generative AI based Azure AI Service, designed to process/ingest content of any types (documents, images, videos, and audio) into a user-defined output format.

Content Understanding offers a streamlined process to reason over large amounts of unstructured data, accelerating time-to-value by generating an output that can be integrated into automation and analytical workflows.

Why process with Content Understanding?

• Simplify and streamline workflows. Azure AI Content Understanding standardizes the extraction of content, structure, and insights from various content types into a unified process.

• Simplify field extraction. Content Understanding's field extraction makes it easier to generate structured output from unstructured content. Define a schema to extract, classify, or generate field values with no complex prompt engineering

• Enhance accuracy. Content Understanding employs multiple AI models to analyze and cross-validate information simultaneously, resulting in more accurate and reliable results.

• Confidence scores & grounding. Content Understanding ensures the accuracy of extracted values while minimizing the cost of human review.

Content Understanding use cases

• Automation. Content Understanding supports automation scenarios by converting unstructured content into structured data, which can be integrated into various workflows and applications. Confidence scores minimize human review and lower costs. For example, automate procurement and payment processes by extracting fields from invoices.

• Search and retrieval augmented generation (RAG). Content Understanding enables ingestion of content of any modality into the search index. The structured output representation improves the relevance for RAG scenarios.

• Analytics and reporting: Content Understanding's extracted field outputs enhance analytics and reporting, allowing businesses to gain valuable insights, conduct deeper analysis, and make informed decisions based on accurate reports.

Applications

Common applications for Content Understanding include:

Application	Description	Quickstart
Post-call analytics	Businesses and call centers can generate insights from call recordings to track key KPIs, improve product experience, generate business insights, create differentiated customer experiences, and answer queries faster and more accurately.	Post-call analytics quickstart
Media asset management	Software and media vendors can use Content Understanding to extract richer, targeted information from videos for media asset management solutions.	Media asset management quickstart
Tax automation	Tax preparation companies can use Content Understanding to generate a unified view of information from various documents and create comprehensive tax returns.	Tax automation quickstart
Chart understanding	Businesses can enhance chart understanding by automating the analysis and interpretation of various types of charts and diagrams using Content Understanding.	Chart understanding quickstart

See Quickstart for more examples.

Components

Component	Description
Analyzer	The analyzer is the core component of Content Understanding. It allows customers to configure content extraction settings and field extraction schema. Once configured, the analyzer consistently applies these settings to process all incoming data.
Content extraction	Content extraction enables users to specify the types of information to be identified and extracted from incoming content. User-specified information includes options such as OCR for text, layout analysis, barcodes, tables, and more, allowing users to focus on the most relevant content elements.
Add-ons	Content Understanding add-ons enhance content extraction by incorporating added elements like barcodes, tables, and detected faces.
Field extraction	Field extraction allows users to define the structure and schema of the desired fields to extract from input files. See service limits for a complete list of field types supported. Fields can be generated via one of the following methods: • Extract: Directly extract values as they appear in the input content, such as dates from receipts or item details from invoices. • Classify: Classify content from a predefined set of categories, such as call sentiment or chart type. • Generate: Generate values freely from input data, such as summarizing an audio conversation or creating scene descriptions from videos.
Grounding source	Content Understanding identifies the specific regions in the content where the value was generated from. Source grounding allows users in automation scenarios to quickly verify the correctness of the field values, leading to higher confidence in the extracted data.
Confidence score	Content Understanding provides confidence scores from 0 to 1 to estimate the reliability of the results. High scores indicate accurate data extraction, enabling straight-through processing in automation workflows.

Responsible AI

Azure AI Content Understanding is designed to guard against processing harmful content, such as graphic violence and gore, hateful speech and bullying, exploitation, abuse, and more. For more information and a full list of prohibited content, see our Transparency note and our Code of Conduct.

Modified Content Filtering

Azure AI Content Understanding now supports turning off content filtering for approved customers. The subscription IDs with approved modified content filtering impacts the Azure AI Content Understanding output.

ข้อสำคัญ

• Apply for modified content filters via this form: Azure OpenAI Limited Access Review: Modified Content Filters.
• For more information, see Content Filtering.

Data privacy and security

Developers using the Content Understanding service should review Microsoft's policies on customer data. For more information, visit our Data, protection and privacy page.

ข้อสำคัญ

If you're using Microsoft products or services to process Biometric Data, you're responsible for: (i) providing notice to data subjects, including with respect to retention periods and destruction; (ii) obtaining consent from data subjects; and (iii) deleting the Biometric Data, all as appropriate, and required under applicable Data Protection Requirements. "Biometric Data" has the meaning articulated in Article 4 of the GDPR and, if applicable, equivalent terms in other data protection requirements. For related information, see Data and Privacy for Face.

Getting started

Our quickstart guides help you quickly start using the Content Understanding service:

• Azure AI Foundry portal Quickstart
• Rest API Quickstart