Edit

What is Azure Form Recognizer?

  • Article
  • 9 minutes to read

This article applies to: Form Recognizer v3.0 checkmark Form Recognizer v3.0. Earlier version: Form Recognizer v2.1

Azure Form Recognizer is a cloud-based Azure Applied AI Service for developers to build intelligent document processing solutions. Form Recognizer applies machine-learning-based optical character recognition (OCR) and document understanding technologies to extract text, tables, structure, and key-value pairs from documents. You can also label and train custom models to automate data extraction from structured, semi-structured, and unstructured documents. To learn more about each model, see the Concepts articles:

Model type Model name
Document analysis models ● Read OCR model
● General document model
● Layout analysis model
Prebuilt models ● W-2 form model
● Invoice model
● Receipt model
● Identity (ID) document model
● Business card model
Custom models ● Custom model
● Composed model

Which Form Recognizer model should I use?

This section will help you decide which Form Recognizer v3.0 supported model you should use for your application:

Type of document Data to extract Document format Your best solution
A generic document like a contract or letter. You want to extract primarily text lines, words, locations, and detected languages. The document is written or printed in a supported language. Read OCR model
A document that includes structural information like a report or study. In addition to text, you need to extract structural information like tables, selection marks, paragraphs, titles, headings, and subheadings. The document is written or printed in a supported language Layout analysis model
A structured or semi-structured document that includes content formatted as fields and values, like a credit application or survey form. You want to extract fields and values including ones not covered by the scenario-specific prebuilt models without having to train a custom model. The form or document is a standardized format commonly used in your business or industry and printed in a supported language. General document model
U.S. W-2 form You want to extract key information such as salary, wages, and taxes withheld from US W2 tax forms. The W-2 document is in United States English (en-US) text. W-2 model
Invoice You want to extract key information such as customer name, billing address, and amount due from invoices. The invoice document is written or printed in a supported language. Invoice model
Receipt You want to extract key information such as merchant name, transaction date, and transaction total from a sales or single-page hotel receipt. The receipt is written or printed in a supported language. Receipt model
Identity document (ID) like a passport or driver's license. You want to extract key information such as first name, last name, and date of birth from US drivers' licenses or international passports. Your ID document is a US driver's license or the biographical page from an international passport (not a visa). Identity document (ID) model
Business card You want to extract key information such as first name, last name, company name, email address, and phone number from business cards. The business card document is in English or Japanese text. Business card model
Mixed-type document(s) You want to extract key-value pairs, selection marks, tables, signature fields, and selected regions not extracted by prebuilt or general document models. You have various documents with structured, semi-structured, and/or unstructured elements. Custom model

Tip

  • If you're still unsure which model to use, try the General Document model to extract key-value pairs.
  • The General Document model is powered by the Read OCR engine to detect text lines, words, locations, and languages.
  • General document also extracts the same data as the document layout model (pages, tables, styles).

Document processing models and development options

Note

The following document understanding models and development options are supported by the Form Recognizer service v3.0.

You can Use Form Recognizer to automate your document processing in applications and workflows, enhance data-driven strategies, and enrich document search capabilities. Use the links in the table to learn more about each model and browse the API references.

Model Description Automation use cases Development options
Read OCR model Extract text lines, words, detected languages, and handwritten style if detected.
  • Contract processing.
  • Financial or medical report processing.
General document model Extract text, tables, structure, and key-value pairs.
  • Key-value pair extraction.
  • Form processing.
  • Survey data collection and analysis.
Layout analysis model Extract text, selection marks, and tables structures, along with their bounding box coordinates, from forms and documents.

Layout API has been updated to a prebuilt model.
  • Document indexing and retrieval by structure.
  • Preprocessing prior to OCR analysis.
Custom model (updated) Extraction and analysis of data from forms and documents specific to distinct business data and use cases.

Custom model API v3.0 supports signature detection for custom template (custom form) models.

Custom model API v3.0 now supports two model types:
  • Identification and compilation of data, unique to your business, impacted by a regulatory change or market event.
  • Identification and analysis of previously overlooked unique data.
 Form Recognizer Studio
  • REST API
  • C# SDK
  • Python SDK
  • Java SDK
  • JavaScript
    		•
    W-2 Form Extract information reported in each box on a W-2 form.
    • Automated tax document management.
    • Mortgage loan application processing.
    Invoice model Automated data processing and extraction of key information from sales invoices.
    • Accounts payable processing.
    • Automated tax recording and reporting.
    Receipt model (updated) Automated data processing and extraction of key information from sales receipts.

    Receipt model v3.0 supports processing of single-page hotel receipts.
    • Expense management.
    • Consumer behavior data analysis.
    • Customer loyalty program.
    • Merchandise return processing.
    • Automated tax recording and reporting.
    Identity document (ID) model (updated) Automated data processing and extraction of key information from US driver's licenses and international passports.

    Prebuilt ID document API supports the extraction of endorsements, restrictions, and vehicle classifications from US driver's licenses.
    • Know your customer (KYC) financial services guidelines compliance.
    • Medical account management.
    • Identity checkpoints and gateways.
    • Hotel registration.
    Business card model Automated data processing and extraction of key information from business cards.
    • Sales lead and marketing management.

    This article applies to: Form Recognizer v2.1 checkmark Form Recognizer v2.1. Later version: Form Recognizer v3.0

    Azure Form Recognizer is a cloud-based Azure Applied AI Service for developers to build intelligent document processing solutions. Form Recognizer applies machine-learning-based optical character recognition (OCR) and document understanding technologies to extract text, tables, structure, and key-value pairs from documents. You can also label and train custom models to automate data extraction from structured, semi-structured, and unstructured documents. To learn more about each model, see the Concepts articles:

    Model type Model name
    Document analysis model ● Layout analysis model
    Prebuilt models ● Invoice model
    ● Receipt model
    ● Identity document (ID) model
    ● Business card model
    Custom models ● Custom model
    ● Composed model

    Which document processing model should I use?

    This section will help you decide which Form Recognizer v2.1 supported model you should use for your application:

    Type of document Data to extract Document format Your best solution
    A document that includes structural information like a report or study. In addition to text, you need to extract structural information like tables and selection marks. The document is written or printed in a supported language Layout analysis model
    Invoice You want to extract key information such as customer name, billing address, and amount due from invoices. The invoice document is written or printed in a supported language. Invoice model
    Receipt You want to extract key information such as merchant name, transaction date, and transaction total from a sales or single-page hotel receipt. The receipt is written or printed in a supported language. Receipt model
    Identity document (ID) like a passport or driver's license. You want to extract key information such as first name, last name, and date of birth from US drivers' licenses or international passports. Your ID document is a US driver's license or the biographical page from an international passport (not a visa). ID document model
    Business card You want to extract key information such as first name, last name, company name, email address, and phone number from business cards. The business card document is in English or Japanese text. Business card model
    Mixed-type document(s) You want to extract key-value pairs, selection marks, tables, signature fields, and selected regions not extracted by prebuilt or general document models. You have various documents with structured, semi-structured, and/or unstructured elements. Custom model

    Form Recognizer models and development options

    Tip

    • For an enhanced experience and advanced model quality, try the Form Recognizer v3.0 Studio.
    • The v3.0 Studio supports any model trained with v2.1 labeled data.
    • You can refer to the API migration guide for detailed information about migrating from v2.1 to v3.0.

    Note

    The following models and development options are supported by the Form Recognizer service v2.1.

    Use the links in the table to learn more about each model and browse the API references:

    Model Description Development options
    Layout analysis Extraction and analysis of text, selection marks, tables, and bounding box coordinates, from forms and documents.
    Custom model Extraction and analysis of data from forms and documents specific to distinct business data and use cases.
    Invoice model Automated data processing and extraction of key information from sales invoices.
    Receipt model Automated data processing and extraction of key information from sales receipts.
    Identity document (ID) model Automated data processing and extraction of key information from US driver's licenses and international passports.
    Business card model Automated data processing and extraction of key information from business cards.

    Data privacy and security

    As with all AI services, developers using the Form Recognizer service should be aware of Microsoft policies on customer data. See our Data, privacy, and security for Form Recognizer page.

    Next steps