Form Recognizer training through API

Question

Hi,
I would like to ask whether it is possible to train a custom model in Form Recognizer using the API. I know that there is an endpoint, https://westus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v3-0-preview-2/operations/BuildDocumentModel, which includes a parameter called "azureBlobSource" for specifying the path to the storage. However, I couldn't find any information on what types of files are accepted in the storage. I assume that files in formats such as JPEG, PNG, and PDF are accepted.

The problem is that, in this case, there is no way to correct how the document was read. If I understand correctly, we fully rely on how the model marks all of the fields. What I would like to do is include my end users in the process of training the model through my web/mobile application.

Is it possible to provide my end users with a similar way of training as the "Label data" section in Form Recognizer, where I can mark fields and tell the model how the document should be read? I would like to generate a COCO file or an Azure ML dataset or any other accepted file format and feed it to the Form Recognizer custom model.

Answer 1

Hi @MarcinZ , Thanks for using Microsoft Q&A Platform.

Yes, it is possible to train a custom model in Form Recognizer using the API. The following document shows Custom model development options. As a best practice, ensure that you use the compatible tools listed here: https://learn.microsoft.com/en-us/azure/applied-ai-services/form-recognizer/concept-custom?view=form-recog-3.0.0&tabs=extraction%2Cclassification#custom-model-development-options

The supported file formats are JPEG/JPG, PNG, BMP, TIFF, and PDF (text-embedded or scanned).

If you want to use manually labeled data, you have to upload the .labels.json and .ocr.json files that correspond to your training documents.

Or you can upload the files in Azure blob storage you can use Form Recognizer studio to label the documents. Then in your blob storage you can find labels.json file and fields.json file.

Once you have your label files, you can include them by calling the training method with the useLabelFile parameter set to true: https://learn.microsoft.com/en-us/azure/applied-ai-services/form-recognizer/how-to-guides/compose-custom-models?view=form-recog-3.0.0&tabs=rest#create-your-custom-models

Here is detailed information on custom models, including supported document files, a quick start to custom modeling, and information on blob storage. Please go through the following documentation: https://learn.microsoft.com/en-us/azure/applied-ai-services/form-recognizer/concept-custom?view=form-recog-3.0.0&tabs=extraction%2Cclassification

I hope this helps.

Regards,
Vasavi