Hello, I am training custom extraction models with Azure Document Intelligence / Form Recognizer. I noticed that the "Read" (e.g. model id: "prebuilt-layout" ) model returns information about paragraphs in it's JSON response under the key: response["analyzeResult"]["paragraphs"] However, running predictions with a custom extraction model does not include paragraph information. In general, the custom extraction model JSON response includes all the other data that a "Read" model response includes (e.g. "pages" , "tables" , and "styles" keys). When will Azure custom extraction models support "paragraphs" information? Thanks, Patrick

Hello @CRAWFORD, PATRICK , Thanks for using Microsoft Q&A Platform. This is a known behavior as this paragraph's information is not yet supported in custom model. However, to extract paragraph/larger span of text using custom model we can do that by region labeling. If you are looking for output similar to the prebuilt layout model, it is recommended to use this model as it is specifically designed for layout analysis and extracting information such as paragraphs, titles, section headings, footnotes, page headers, page footers, and page numbers. We don't have any ETA on this paragraph support but will definitely share your feedback to the product team. I hope this helps. Regards, Vasavi -Please kindly accept the answer and vote 'yes' if you feel helpful to support the community, thanks.

Document Intelligence / Form Recognizer Custom extraction model results does not include paragraph information

Accepted answer

VasaviLankipalle-MSFT 17,641 Reputation points

2023-08-28T21:09:01.1133333+00:00

Hello @CRAWFORD, PATRICK , Thanks for using Microsoft Q&A Platform.

This is a known behavior as this paragraph's information is not yet supported in custom model. However, to extract paragraph/larger span of text using custom model we can do that by region labeling.

If you are looking for output similar to the prebuilt layout model, it is recommended to use this model as it is specifically designed for layout analysis and extracting information such as paragraphs, titles, section headings, footnotes, page headers, page footers, and page numbers.

We don't have any ETA on this paragraph support but will definitely share your feedback to the product team.

I hope this helps.

Regards,
Vasavi

-Please kindly accept the answer and vote 'yes' if you feel helpful to support the community, thanks.
Please sign in to rate this answer.
CRAWFORD, PATRICK 25 Reputation points

2023-08-28T22:24:36.0133333+00:00

This is not what I am asking. I do not want to label regions.

I am looking for output similar to the prebuilt layout model. I am aware that the prebuilt layout model will extract information such as paragraphs.

Custom models already give a response which includes some of the information of the prebuilt layout model. However, custom models do not give all the information of the prebuilt layout model. To me, it seems likely that under the hood of a custom model, the prebuilt layout model is being run and then ensembled with a custom named-entity recognition model. The following led me to this conclusion:

When performing labeling on the Form Recognizer studio, you must first analyze documents with the prebuilt layout model before they are available for to be labeled. The prebuilt layout model prediction responses are stored as .ocr.json files in the underlying blob storage of the Form Recognizer labeling project

The JSON responses from predicting with a custom model and the JSON response from predicting with a prebuilt layout model are eerily similar; they already contain many of the same keys "pages", "tables", and "styles"

In light of this, could you help me understand why the "paragraphs" information is not also included in the JSON response to predicting with a custom model?

VasaviLankipalle-MSFT 17,641 Reputation points

2023-08-28T23:16:11.4166667+00:00

Hello @CRAWFORD, PATRICK , thanks for sharing the detailed information. I will try to help you understand the product limitation. As I mentioned earlier the paragraphs are not supported so the Json result is also different in the custom model. In the layout model, the paragraph roles that are supported are listed and other fields are mentioned so you can observe the Json result similar to this.

The custom model is designed to extract specific fields and data from documents based on requirements, while the prebuilt layout model is designed to provide a comprehensive layout analysis of the document: https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/concept-custom?view=doc-intel-3.1.0#custom-model-extraction-summary

CRAWFORD, PATRICK 25 Reputation points

2023-08-29T13:20:14.5133333+00:00

I understand now, thank you for clarifying!
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Document Intelligence / Form Recognizer Custom extraction model results does not include paragraph information

0 additional answers

Your answer