Azure Form Recognizer receipt model

This article applies to: Form Recognizer v3.0 checkmark Form Recognizer v3.0. Earlier version: Form Recognizer v2.1

This article applies to: Form Recognizer v2.1 checkmark Form Recognizer v2.1. Later version: Form Recognizer v3.0

The Form Recognizer receipt model combines powerful Optical Character Recognition (OCR) capabilities with deep learning models to analyze and extract key information from sales receipts. Receipts can be of various formats and quality including printed and handwritten receipts. The API extracts key information such as merchant name, merchant phone number, transaction date, tax, and transaction total and returns structured JSON data.

Receipt data extraction

Receipt digitization is the process of converting scanned receipts into digital form for downstream processing. Azure Form Recognizer OCR-powered receipt data extraction helps to automate the conversion and save time and effort. The output from the receipt data extraction is used for accounts payable and receivables automation, sales data analytics, and other business scenarios.

Sample receipt processed with Form Recognizer Studio:

Screenshot of a sample receipt processed in the Form Recognizer Studio.

Sample receipt processed with Form Recognizer Sample Labeling tool:

Screenshot of a sample receipt processed with the Form Sample Labeling tool.

Development options

The following tools are supported by Form Recognizer v3.0:

Feature Resources Model ID
Receipt model prebuilt-receipt

The following tools are supported by Form Recognizer v2.1:

Feature Resources
Receipt model

Input requirements

  • For best results, provide one clear photo or high-quality scan per document.

  • Supported file formats:

    Model PDF Image:
    JPEG/JPG, PNG, BMP, and TIFF
    Microsoft Office:
    Word (DOCX), Excel (XLS), PowerPoint (PPT), and HTML
    Read REST API version
    2022/06/30-preview
    Layout
    General Document
    Prebuilt
    Custom

    ✱ Microsoft Office files are currently not supported for other models or versions.

  • For PDF and TIFF, up to 2000 pages can be processed (with a free tier subscription, only the first two pages are processed).

  • The file size for analyzing documents must be less than 500 MB for paid (S0) tier and 4 MB for free (F0) tier.

  • Image dimensions must be between 50 x 50 pixels and 10,000 px x 10,000 pixels.

  • PDF dimensions are up to 17 x 17 inches, corresponding to Legal or A3 paper size, or smaller.

  • If your PDFs are password-locked, you must remove the lock before submission.

  • The minimum height of the text to be extracted is 12 pixels for a 1024 x 768 pixel image. This dimension corresponds to about 8-point text at 150 dots per inch (DPI).

  • For custom model training, the maximum number of pages for training data is 500 for the custom template model and 50,000 for the custom neural model.

  • For custom model training, the total size of training data is 50 MB for template model and 1G-MB for the neural model.

  • Supported file formats: JPEG, PNG, PDF, and TIFF
  • For PDF and TIFF, up to 2000 pages are processed. For free tier subscribers, only the first two pages are processed.
  • The file size must be less than 50 MB and dimensions at least 50 x 50 pixels and at most 10,000 x 10,000 pixels.

Try receipt data extraction

See how data, including time and date of transactions, merchant information, and amount totals, is extracted from receipts. You'll need the following resources:

  • An Azure subscription—you can create one for free

  • A Form Recognizer instance in the Azure portal. You can use the free pricing tier (F0) to try the service. After your resource deploys, select Go to resource to get your key and endpoint.

Screenshot: keys and endpoint location in the Azure portal.

Form Recognizer Studio

Note

Form Recognizer studio is available with the v3.0 API.

  1. On the Form Recognizer Studio home page, select Receipts

  2. You can analyze the sample receipt or select the + Add button to upload your own sample.

  3. Select the Analyze button:

    Screenshot: analyze receipt menu.

Form Recognizer Sample Labeling tool

  1. Navigate to the Form Recognizer Sample Tool.

  2. On the sample tool home page, select the Use prebuilt model to get data tile.

    Screenshot of the layout model analyze results process.

  3. Select the Form Type to analyze from the dropdown menu.

  4. Choose a URL for the file you would like to analyze from the below options:

  5. In the Source field, select URL from the dropdown menu, paste the selected URL, and select the Fetch button.

    Screenshot of source location dropdown menu.

  6. In the Form Recognizer service endpoint field, paste the endpoint that you obtained with your Form Recognizer subscription.

  7. In the key field, paste the key you obtained from your Form Recognizer resource.

    Screenshot of the select-form-type dropdown menu.

  8. Select Run analysis. The Form Recognizer Sample Labeling tool will call the Analyze Prebuilt API and analyze the document.

  9. View the results - see the key-value pairs extracted, line items, highlighted text extracted and tables detected.

    Screenshot of the layout model analyze results operation.

Note

The Sample Labeling tool does not support the BMP file format. This is a limitation of the tool not the Form Recognizer Service.

Supported languages and locales v3.0

Note

It's not necessary to specify a locale. This is an optional parameter. The Form Recognizer deep-learning technology will auto-detect the language of the text in your image.

The receipt model supports all English receipts and the following locales:

Language Locale code
English (Australia) en-au
English (Canada) en-ca
English (United Kingdom) en-gb
English (India en-in
English (United States) en-us
French 'fr'
Spanish es

Supported languages and locales v2.1

Note

It's not necessary to specify a locale. This is an optional parameter. The Form Recognizer deep-learning technology will auto-detect the language of the text in your image.

Model Language—Locale code Default
Receipt
  • English (United States)—en-US
  • English (Australia)—en-AU
  • English (Canada)—en-CA
  • English (United Kingdom)—en-GB
  • English (India)—en-IN
Autodetected

Field extraction

Name Type Description Standardized output
ReceiptType String Type of sales receipt Itemized
MerchantName String Name of the merchant issuing the receipt
MerchantPhoneNumber phoneNumber Listed phone number of merchant +1 xxx xxx xxxx
MerchantAddress String Listed address of merchant
TransactionDate Date Date the receipt was issued yyyy-mm-dd
TransactionTime Time Time the receipt was issued hh-mm-ss (24-hour)
Total Number (USD) Full transaction total of receipt Two-decimal float
Subtotal Number (USD) Subtotal of receipt, often before taxes are applied Two-decimal float
Tax Number (USD) Total tax on receipt (often sales tax or equivalent). Renamed to "TotalTax" in 2022-06-30 version. Two-decimal float
Tip Number (USD) Tip included by buyer Two-decimal float
Items Array of objects Extracted line items, with name, quantity, unit price, and total price extracted
Name String Item description. Renamed to "Description" in 2022-06-30 version.
Quantity Number Quantity of each item Two-decimal float
Price Number Individual price of each item unit Two-decimal float
TotalPrice Number Total price of line item Two-decimal float

Form Recognizer v3.0 introduces several new features and capabilities. The Receipt model supports single-page hotel receipt processing.

Hotel receipt field extraction

Name Type Description Standardized output
ArrivalDate Date Date of arrival yyyy-mm-dd
Currency Currency Currency unit of receipt amounts. For example USD, EUR, or MIXED if multiple values are found
DepartureDate Date Date of departure yyyy-mm-dd
Items Array
Items.*.Category String Item category, for example, Room, Tax, etc.
Items.*.Date Date Item date yyyy-mm-dd
Items.*.Description String Item description
Items.*.TotalPrice Number Item total price Two-decimal float
MerchantAddress String Listed address of merchant
MerchantAliases Array
MerchantAliases.* String Alternative name of merchant
MerchantName String Name of the merchant issuing the receipt
MerchantPhoneNumber phoneNumber Listed phone number of merchant +1 xxx xxx xxxx
ReceiptType String Type of receipt, for example, Hotel, Itemized
Total Number Full transaction total of receipt Two-decimal float

Hotel receipt supported languages and locales

Model Language—Locale code Default
Receipt (hotel)
  • English (United States)—en-US
English (United States)—en-US

Migration guide and REST API v3.0

Next steps