What is Azure AI Document Intelligence?

Important

  • Document Intelligence public preview releases provide early access to features that are in active development.
  • Features, approaches, and processes may change, prior to General Availability (GA), based on user feedback.
  • The public preview version of Document Intelligence client libraries default to REST API version 2024-02-29-preview.
  • Public preview version 2024-02-29-preview is currently only available in the following Azure regions:
  • East US
  • West US2
  • West Europe

This content applies to: checkmark v4.0 (preview) | Previous versions: blue-checkmark v3.1 (GA) blue-checkmark v3.0 (GA) blue-checkmark v2.1 (GA)

This content applies to: checkmark v3.1 (GA) | Latest version: purple-checkmark v4.0 (preview) | Previous versions: blue-checkmark v3.0 blue-checkmark v2.1

This content applies to: checkmark v3.0 (GA) | Latest versions: purple-checkmark v4.0 (preview) purple-checkmark v3.1 (preview) | Previous version: blue-checkmark v2.1

This content applies to: checkmark v2.1 | Latest version: blue-checkmark v4.0 (preview)

Note

Form Recognizer is now Azure AI Document Intelligence!

  • As of July 2023, Azure AI services encompass all of what were previously known as Cognitive Services and Azure Applied AI Services.
  • There are no changes to pricing.
  • The names Cognitive Services and Azure Applied AI continue to be used in Azure billing, cost analysis, price list, and price APIs.
  • There are no breaking changes to application programming interfaces (APIs) or SDKs prior to and including v3.1. Starting from v4.0, APIs and SDKs are updated to Document Intelligence.
  • Some platforms are still awaiting the renaming update. All mention of Form Recognizer or Document Intelligence in our documentation refers to the same Azure service.

Azure AI Document Intelligence is a cloud-based Azure AI service that enables you to build intelligent document processing solutions. Massive amounts of data, spanning a wide variety of data types, are stored in forms and documents. Document Intelligence enables you to effectively manage the velocity at which data is collected and processed and is key to improved operations, informed data-driven decisions, and enlightened innovation.

| ✔️ Document analysis models | ✔️ Prebuilt models | ✔️ Custom models |

Document analysis models

Document analysis models enable text extraction from forms and documents and return structured business-ready content ready for your organization's action, use, or progress.


Read | Extract printed
and handwritten text.


Layout | Extract text, tables,
and document structure.


Read | Extract printed
and handwritten text.


Layout | Extract text, tables,
and document structure.


General document | Extract text,
structure, and key-value pairs.

Prebuilt models

Prebuilt models enable you to add intelligent document processing to your apps and flows without having to train and build your own models.


Invoice | Extract customer
and vendor details.


Receipt | Extract sales
transaction details.


Identity | Extract identification
and verification details.


1003 EULA | Extract mortgage details.


Form 1008 | Extract mortgage details.


Closing Disclosure | Extract mortgage details.


Health Insurance card | Extract health
insurance details.


Contract | Extract agreement
and party details.


Credit/Debit card | Extract information from bank cards.


Marriage Certificate | Extract information from Marriage certificates.


US Tax W-2 form | Extract taxable
compensation details.


US Tax 1098 form | Extract mortgage interest details.


US Tax 1098-E form | Extract student loan interest details.


US Tax 1098-T form | Extract qualified tuition details.


US Tax 1099 form | Extract information from variations of the 1099 form.


US Tax 1040 form | Extract information from variations of the 1040 form.


Invoice | Extract customer
and vendor details.


Receipt | Extract sales
transaction details.


Identity | Extract identification
and verification details.


Health Insurance card | Extract health insurance details.


Business card | Extract business contact details.


Contract | Extract agreement
and party details.


US Tax W-2 form | Extract taxable
compensation details.


US Tax 1098 form | Extract mortgage interest details.


US Tax 1098-E form | Extract student loan interest details.


US Tax 1098-T form | Extract qualified tuition details.

Custom models

  • Custom models are trained using your labeled datasets to extract distinct data from forms and documents, specific to your use cases.
  • Standalone custom models can be combined to create composed models.
  • Extraction models
    ✔️ Custom extraction models are trained to extract labeled fields from documents.


Custom template | Extract data from static layouts.


Custom neural | Extract data from mixed-type documents.


Custom composed | Extract data using a collection of models.

  • Classification model
    ✔️ Custom classifiers identify document types before invoking an extraction model.


Custom classifier | Identify designated document types (classes)
before invoking an extraction model.

Add-on capabilities

Document Intelligence supports optional features that can be enabled and disabled depending on the document extraction scenario. The following add-on capabilities are available for 2023-07-31 (GA) and later releases:

Document Intelligence supports optional features that can be enabled and disabled depending on the document extraction scenario. The following add-on capabilities areavailable for2024-02-29-preview, 2023-10-31-preview, and later releases:

Analysis features

Model ID Content Extraction Query fields Paragraphs Paragraph Roles Selection Marks Tables Key-Value Pairs Languages Barcodes Document Analysis Formulas* Style Font* High Resolution*
prebuilt-read O O O O O
prebuilt-layout O O O O O
prebuilt-document O O O O O
prebuilt-businessCard
prebuilt-contract O O O O O
prebuilt-healthInsuranceCard.us O O O O O
prebuilt-idDocument O O O O O
prebuilt-invoice O O O O O O
prebuilt-receipt O O O O O
prebuilt-marriageCertificate.us O O O O O
prebuilt-creditCard O O O O O
prebuilt-mortgage.us.1003 O O O O O
prebuilt-mortgage.us.1008 O O O O O
prebuilt-mortgage.us.closingDisclosure O O O O O
prebuilt-tax.us.w2 O O O O O
prebuilt-tax.us.1098 O O O O O
prebuilt-tax.us.1098E O O O O O
prebuilt-tax.us.1098T O O O O O
prebuilt-tax.us.1099(variations) O O O O O
prebuilt-tax.us.1040(variations) O O O O O
{ customModelName } O O O O O

✓ - Enabled
O - Optional
* - Premium features incur extra costs

Models and development options

Note

The following document understanding models and development options are supported by the Document Intelligence service v3.0.

You can use Document Intelligence to automate document processing in applications and workflows, enhance data-driven strategies, and enrich document search capabilities. Use the links in the table to learn more about each model and browse development options.

Read

Screenshot of Read model analysis using Document Intelligence Studio.

Model ID Description Automation use cases Development options
prebuilt-read ● Extract text from documents.
Data extraction
● Digitizing any document.
● Compliance and auditing.
● Processing handwritten notes before translation.
Document Intelligence Studio
REST API
C# SDK
Python SDK
Java SDK
JavaScript

Layout

Screenshot of the layout model analysis using Document Intelligence Studio.

Model ID Description Automation use cases Development options
prebuilt-layout ● Extract text and layout information from documents.
Data extraction
● Document indexing and retrieval by structure.
● Financial and medical report analysis.
Document Intelligence Studio
REST API
C# SDK
Python SDK
Java SDK
JavaScript

General document (deprecated in 2023-10-31-preview)

Screenshot of General Document model analysis using Document Intelligence Studio.

Model ID Description Automation use cases Development options
prebuilt-document ● Extract text,layout, and key-value pairs from documents.
Data and field extraction
● Key-value pair extraction.
● Form processing.
● Survey data collection and analysis.
Document Intelligence Studio
REST API

Invoice

Screenshot of Invoice model analysis using Document Intelligence Studio.

Model ID Description Automation use cases Development options
prebuilt-invoice ● Extract key information from invoices.
Data and field extraction
● Accounts payable processing.
● Automated tax recording and reporting.
Document Intelligence Studio
REST API
C# SDK
Python SDK
Java SDK
JavaScript

Receipt

Screenshot of Receipt model analysis using Document Intelligence Studio.

Model ID Description Automation use cases Development options
prebuilt-receipt ● Extract key information from receipts.
Data and field extraction
● Receipt model v3.0 supports processing of single-page hotel receipts.
● Expense management.
● Consumer behavior data analysis.
● Customer loyalty program.
● Merchandise return processing.
● Automated tax recording and reporting.
Document Intelligence Studio
REST API
C# SDK
Python SDK
Java SDK
JavaScript

Identity (ID)

Screenshot of Identity (ID) Document model analysis using Document Intelligence Studio.

Model ID Description Automation use cases Development options
prebuilt-idDocument ● Extract key information from passports and ID cards.
Document types
● Extract endorsements, restrictions, and vehicle classifications from US driver's licenses.
● Know your customer (KYC) financial services guidelines compliance.
● Medical account management.
● Identity checkpoints and gateways.
● Hotel registration.
Document Intelligence Studio
REST API
C# SDK
Python SDK
Java SDK
JavaScript

Health insurance card

Screenshot of Health insurance card model analysis using Document Intelligence Studio.

Model ID Description Automation use cases Development options
prebuilt-healthInsuranceCard.us ● Extract key information from US health insurance cards.
Data and field extraction
● Coverage and eligibility verification.
● Predictive modeling.
● Value-based analytics.
Document Intelligence Studio
REST API
C# SDK
Python SDK
Java SDK
JavaScript

Contract model

Screenshot of Contract model extraction using Document Intelligence Studio.

Model ID Description Development options
prebuilt-contract Extract contract agreement and party details. Document Intelligence Studio
REST API

US Tax W-2 model

Screenshot of W-2 model analysis using Document Intelligence Studio.

Model ID Description Automation use cases Development options
prebuilt-tax.us.W-2 ● Extract key information from IRS US W2 tax forms (year 2018-2021).
Data and field extraction
● Automated tax document management.
● Mortgage loan application processing.
Document Intelligence Studio
REST API
C# SDK
Python SDK
Java SDK
JavaScript

US tax 1098 form

Screenshot of US 1098 tax form analyzed in the Document Intelligence Studio.

Model ID Description Development options
prebuilt-tax.us.1098 Extract mortgage interest information and details. Document Intelligence Studio
REST API

US tax 1098-E form

Screenshot of US 1098-E tax form analyzed in the Document Intelligence Studio.

Model ID Description Development options
prebuilt-tax.us.1098E Extract student loan information and details. Document Intelligence Studio
REST API

US tax 1098-T form

Screenshot of US 1098-T tax form analyzed in the Document Intelligence Studio.

Model ID Description Development options
prebuilt-tax.us.1098T Extract tuition information and details. Document Intelligence Studio
REST API

US tax 1099 (and Variations) form

Screenshot of US 1099 tax form analyzed in the Document Intelligence Studio.

Model ID Description Development options
prebuilt-tax.us.1099(Variations) Extract information from 1099-form variations. Document Intelligence Studio
REST API

Business card

Screenshot of Business card model analysis using Document Intelligence Studio.

Model ID Description Automation use cases Development options
prebuilt-businessCard ● Extract key information from business cards.
Data and field extraction
● Sales lead and marketing management. Document Intelligence Studio
REST API
C# SDK
Python SDK
Java SDK
JavaScript

Custom model overview

Screenshot of Custom model training using Document Intelligence Studio.

About Description Automation use cases Development options
Custom model Extracts information from forms and documents into structured data based on a model created from a set of representative training document sets. Extract distinct data from forms and documents specific to your business and use cases. Document Intelligence Studio
REST API
C# SDK
Java SDK
JavaScript SDK
Python SDK

Custom template

Screenshot of Custom Template model analysis using Document Intelligence Studio.

Note

To train a custom template model, set the buildMode property to template. For more information, see Training a template model

About Description Automation use cases Development options
Custom Template model The custom template model extracts labeled values and fields from structured and semi-structured documents.
Extract key data from highly structured documents with defined visual templates or common visual layouts, forms. Document Intelligence Studio
REST API
C# SDK
Python SDK
Java SDK
JavaScript SDK

Custom neural

Screenshot of Custom Neural model analysis using Document Intelligence Studio.

Note

To train a custom neural model, set the buildMode property to neural. For more information, see Training a neural model

About Description Automation use cases Development options
Custom Neural model The custom neural model is used to extract labeled data from structured (surveys, questionnaires), semi-structured (invoices, purchase orders), and unstructured documents (contracts, letters). Extract text data, checkboxes, and tabular fields from structured and unstructured documents. Document Intelligence Studio
REST API
C# SDK
Java SDK
JavaScript SDK
Python SDK

Custom composed

Screenshot of Composed Custom model list in Document Intelligence Studio.

About Description Automation use cases Development options
Composed custom models A composed model is created by taking a collection of custom models and assigning them to a single model built from your form types. Useful when you train several models and want to group them to analyze similar form types like purchase orders. Document Intelligence Studio
REST API
C# SDK
Java SDK
JavaScript SDK
Python SDK

Custom classification model

{alt-text}

About Description Automation use cases Development options
Composed classification model Custom classification models combine layout and language features to detect, identify, and classify documents within an input file. ● A loan application packaged containing application form, payslip, and, bank statement.
● A collection of scanned invoices.
Document Intelligence Studio
REST API

Azure AI Document Intelligence is a cloud-based Azure AI service for developers to build intelligent document processing solutions. Document Intelligence applies machine-learning-based optical character recognition (OCR) and document understanding technologies to extract text, tables, structure, and key-value pairs from documents. You can also label and train custom models to automate data extraction from structured, semi-structured, and unstructured documents. To learn more about each model, see the Concepts articles:

Model type Model name
Document analysis model Layout analysis model
Prebuilt models Invoice model
Receipt model
Identity document (ID) model
Business card model
Custom models Custom model
Composed model

This content applies to: checkmark v2.1 | Latest version: blue-checkmark v4.0 (preview)

Document Intelligence models and development options

Tip

  • For an enhanced experience and advanced model quality, try the Document Intelligence v3.0 Studio.
  • The v3.0 Studio supports any model trained with v2.1 labeled data.
  • You can refer to the API migration guide for detailed information about migrating from v2.1 to v3.0.

Note

The following models and development options are supported by the Document Intelligence service v2.1.

Use the links in the table to learn more about each model and browse the API references:

Model Description Development options
Layout analysis Extraction and analysis of text, selection marks, tables, and bounding box coordinates, from forms and documents. Document Intelligence labeling tool
REST API
Client-library SDK
Document Intelligence Docker container
Custom model Extraction and analysis of data from forms and documents specific to distinct business data and use cases. Document Intelligence labeling tool
REST API
Sample Labeling Tool
Document Intelligence Docker container
Invoice model Automated data processing and extraction of key information from sales invoices. Document Intelligence labeling tool
REST API
Client-library SDK
Document Intelligence Docker container
Receipt model Automated data processing and extraction of key information from sales receipts. Document Intelligence labeling tool
REST API
Client-library SDK
Document Intelligence Docker container
Identity document (ID) model Automated data processing and extraction of key information from US driver's licenses and international passports. Document Intelligence labeling tool
REST API
Client-library SDK
Document Intelligence Docker container
Business card model Automated data processing and extraction of key information from business cards. Document Intelligence labeling tool
REST API
Client-library SDK
Document Intelligence Docker container

Data privacy and security

As with all AI services, developers using the Document Intelligence service should be aware of Microsoft policies on customer data. See our Data, privacy, and security for Document Intelligence page.

Next steps