Edit

Azure Form Recognizer receipt model

  • Article
  • 9 minutes to read

This article applies to: Form Recognizer v3.0 checkmark Form Recognizer v3.0. Earlier version: Form Recognizer v2.1

This article applies to: Form Recognizer v2.1 checkmark Form Recognizer v2.1. Later version: Form Recognizer v3.0

The Form Recognizer receipt model combines powerful Optical Character Recognition (OCR) capabilities with deep learning models to analyze and extract key information from sales receipts. Receipts can be of various formats and quality including printed and handwritten receipts. The API extracts key information such as merchant name, merchant phone number, transaction date, tax, and transaction total and returns structured JSON data.

Receipt data extraction

Receipt digitization is the process of converting scanned receipts into digital form for downstream processing. Azure Form Recognizer OCR-powered receipt data extraction helps to automate the conversion and save time and effort.

Sample receipt processed with Form Recognizer Studio:

Screenshot of a sample receipt processed in the Form Recognizer Studio.

Sample receipt processed with Form Recognizer Sample Labeling tool:

Screenshot of a sample receipt processed with the Form Sample Labeling tool.

Development options

Form Recognizer v3.0 Supports the following tools:

Feature Resources Model ID
Receipt model prebuilt-receipt

Form Recognizer v2.1 supports the following tools:

Feature Resources
Receipt model

Input requirements

  • For best results, provide one clear photo or high-quality scan per document.

  • Supported file formats:

    Model PDF Image:
    JPEG/JPG, PNG, BMP, and TIFF    		 Microsoft Office:
    Word (DOCX), Excel (XLS), PowerPoint (PPT), and HTML
    Read ✔ ✔ ✱ REST API version
    2022/06/30-preview
    Layout ✔ ✔
    General Document ✔ ✔
    Prebuilt ✔ ✔
    Custom ✔ ✔

    ✱ Microsoft Office files are currently not supported for other models or versions.

  • For PDF and TIFF, up to 2000 pages can be processed (with a free tier subscription, only the first two pages are processed).

  • The file size for analyzing documents must be less than 500 MB for paid (S0) tier and 4 MB for free (F0) tier.

  • Image dimensions must be between 50 x 50 pixels and 10,000 px x 10,000 pixels.

  • PDF dimensions are up to 17 x 17 inches, corresponding to Legal or A3 paper size, or smaller.

  • If your PDFs are password-locked, you must remove the lock before submission.

  • The minimum height of the text to be extracted is 12 pixels for a 1024 x 768 pixel image. This dimension corresponds to about 8-point text at 150 dots per inch (DPI).

  • For custom model training, the maximum number of pages for training data is 500 for the custom template model and 50,000 for the custom neural model.

  • For custom extraction model training, the total size of training data is 50 MB for template model and 1G-MB for the neural model.

  • For custom classification model training, the total size of training data is 1GB with a maximum of 10,000 pages.

  • Supported file formats: JPEG, PNG, PDF, and TIFF
  • For PDF and TIFF, Form Recognizer can process up to 2000 pages for standard tier subscribers or only the first two pages for free-tier subscribers.
  • The file size must be less than 50 MB and dimensions at least 50 x 50 pixels and at most 10,000 x 10,000 pixels.

Try receipt data extraction

See how Form Recognizer extracts data, including time and date of transactions, merchant information, and amount totals from receipts. You need the following resources:

  • An Azure subscription—you can create one for free

  • A Form Recognizer instance in the Azure portal. You can use the free pricing tier (F0) to try the service. After your resource deploys, select Go to resource to get your key and endpoint.

Screenshot: keys and endpoint location in the Azure portal.

Form Recognizer Studio

Note

Form Recognizer studio is available with the v3.0 API.

  1. On the Form Recognizer Studio home page, select Receipts

  2. You can analyze the sample receipt or select the + Add button to upload your own sample.

  3. Select the Analyze button:

    Screenshot: analyze receipt menu.

    Try Form Recognizer Studio

Form Recognizer Sample Labeling tool

  1. Navigate to the Form Recognizer Sample Tool.

  2. On the sample tool home page, select the Use prebuilt model to get data tile.

    Screenshot of the layout model analyze results process.

  3. Select the Form Type to analyze from the dropdown menu.

  4. Choose a URL for the file you would like to analyze from the below options:

  5. In the Source field, select URL from the dropdown menu, paste the selected URL, and select the Fetch button.

    Screenshot of source location dropdown menu.

  6. In the Form Recognizer service endpoint field, paste the endpoint that you obtained with your Form Recognizer subscription.

  7. In the key field, paste the key you obtained from your Form Recognizer resource.

    Screenshot of the select-form-type dropdown menu.

  8. Select Run analysis. The Form Recognizer Sample Labeling tool calls the Analyze Prebuilt API and analyze the document.

  9. View the results - see the key-value pairs extracted, line items, highlighted text extracted and tables detected.

    Screenshot of the layout model analyze results operation.

Note

The Sample Labeling tool does not support the BMP file format. This is a limitation of the tool not the Form Recognizer Service.

Supported languages and locales

Note

Form Recognizer auto-detects language and locale data.

Thermal receipts (retail, meal, parking, etc.)

Supported Languages Details
English United States (en-US), Australia (en-AU), Canada (en-CA), United Kingdom (en-GB), India (en-IN), United Arab Emirates (en-AE)
Croatian Croatia (hr-HR)
Czech Czechia (cs-CZ)
Danish Denmark (da-DK)
Dutch Netherlands (nl-NL)
Finnish Finland (fi-FI)
French Canada (fr-CA), France (fr-FR)
German Germany (de-DE)
Hungarian Hungary (hu-HU)
Italian Italy (it-IT)
Japanese Japan (ja-JP)
Latvian Latvia (lv-LV)
Lithuanian Lithuania (lt-LT)
Norwegian Norway (no-NO)
Portuguese Brazil (pt-BR), Portugal (pt-PT)
Spanish Spain (es-ES)
Swedish Sweden (sv-SE)
Vietnamese Vietnam (vi-VN)

Hotel receipts

Supported Languages Details
English United States (en-US)
French France (fr-FR)
German Germany (de-DE)
Italian Italy (it-IT)
Japanese Japan (ja-JP)
Portuguese Portugal (pt-PT)
Spanish Spain (es-ES)

Supported languages and locales v2.1

Note

It's not necessary to specify a locale. This is an optional parameter. The Form Recognizer deep-learning technology will auto-detect the language of the text in your image.

Model Language—Locale code Default
Receipt
  • English (United States)—en-US
  • English (Australia)—en-AU
  • English (Canada)—en-CA
  • English (United Kingdom)—en-GB
  • English (India)—en-IN
 Autodetected

Field extraction

Name Type Description Standardized output
ReceiptType String Type of sales receipt Itemized
MerchantName String Name of the merchant issuing the receipt
MerchantPhoneNumber phoneNumber Listed phone number of merchant +1 xxx xxx xxxx
MerchantAddress String Listed address of merchant
TransactionDate Date Date the receipt was issued yyyy-mm-dd
TransactionTime Time Time the receipt was issued hh-mm-ss (24-hour)
Total Number (USD) Full transaction total of receipt Two-decimal float
Subtotal Number (USD) Subtotal of receipt, often before taxes are applied Two-decimal float
Tax Number (USD) Total tax on receipt (often sales tax or equivalent). Renamed to "TotalTax" in 2022-06-30 version. Two-decimal float
Tip Number (USD) Tip included by buyer Two-decimal float
Items Array of objects Extracted line items, with name, quantity, unit price, and total price extracted
Name String Item description. Renamed to "Description" in 2022-06-30 version.
Quantity Number Quantity of each item Two-decimal float
Price Number Individual price of each item unit Two-decimal float
TotalPrice Number Total price of line item Two-decimal float

Form Recognizer v3.0 introduces several new features and capabilities. In addition to thermal receipts, the Receipt model supports single-page hotel receipt processing and tax detail extraction for all receipt types.

Thermal receipts (receipt, receipt.retailMeal, receipt.creditCard, receipt.gas, receipt.parking)

Field Type Description Example
MerchantName string Name of the merchant issuing the receipt Contoso
MerchantPhoneNumber phoneNumber Listed phone number of merchant 987-654-3210
MerchantAddress address Listed address of merchant 123 Main St. Redmond WA 98052
Total number Full transaction total of receipt $14.34
TransactionDate date Date the receipt was issued June 06, 2019
TransactionTime time Time the receipt was issued 4:49 PM
Subtotal number Subtotal of receipt, often before taxes are applied $12.34
TotalTax number Tax on receipt, often sales tax or equivalent $2.00
Tip number Tip included by buyer $1.00
Items array
Items.* object Extracted line item 1
Surface Pro 6
$999.00
$999.00
Items.*.TotalPrice number Total price of line item $999.00
Items.*.Description string Item description Surface Pro 6
Items.*.Quantity number Quantity of each item 1
Items.*.Price number Individual price of each item unit $999.00
Items.*.ProductCode string Product code, product number, or SKU associated with the specific line item A123
Items.*.QuantityUnit string Quantity unit of each item
TaxDetails array
TaxDetails.* object Extracted line item 1
Surface Pro 6
$999.00
$999.00
TaxDetails.*.Amount currency The amount of the tax detail $999.00

Hotel receipts (receipt.hotel)

Field Type Description Example
MerchantName string Name of the merchant issuing the receipt Contoso
MerchantPhoneNumber phoneNumber Listed phone number of merchant 987-654-3210
MerchantAddress address Listed address of merchant 123 Main St. Redmond WA 98052
Total number Full transaction total of receipt $14.34
ArrivalDate date Date of arrival 27Mar21
DepartureDate date Date of departure 28Mar21
Currency string Currency unit of receipt amounts (ISO 4217), or 'MIXED' if multiple values are found USD
MerchantAliases array
MerchantAliases.* string Alternative name of merchant Contoso (R)
Items array
Items.* object Extracted line item 1
Surface Pro 6
$999.00
$999.00
Items.*.TotalPrice number Total price of line item $999.00
Items.*.Description string Item description Room Charge
Items.*.Date date Item date 27Mar21
Items.*.Category string Item category Room

Migration guide and REST API v3.0

Next steps