Combining Form Recognizer with OpenAI for Improved Table Extraction - Seeking Examples/Documentation

Millns, Bailey 0

Hello community,

I'm currently working on a project that involves extracting tables from UK financial statements. I have been using Azure Form Recognizer, but I'm encountering challenges due to the diverse layouts and inconsistent table naming across documents. The supervised extraction isn't as efficient as I hoped it would be.

In an effort to overcome these obstacles, I'm considering combining the Form Recognizer with OpenAI. My idea is to use Form Recognizer for the initial extraction process (unsupervised), and then employ OpenAI to find the specific tables I need.

Has anyone attempted a similar solution before? I'm seeking documentation or examples of Azure implementations where Form Recognizer and OpenAI have been used in tandem, specifically for table extraction from documents. Any guidance on how to train or configure this kind of system would be highly appreciated.

Thank you

Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Millns, Bailey 0 Reputation points

2023-07-19T18:30:20.2266667+00:00

Thank you for your response. My issue with table extraction is specifically related to inconsistencies in the layout of the same type of table across different financial documents. I am trying to extract the income table from UK financial statements, and I've found that the layout varies from one document to another.

Here are two examples for reference:

I did try training a custom extraction model as per your suggestion. However, I encountered errors due to incorrect labeling of the documents. At this stage, I am reluctant to spend more time labeling numerous documents for the model.

My preference is to use unsupervised extraction to pull all elements from the documents, and then leverage OpenAI to filter and identify the income table. I'm also open to using Azure Cognitive Search if it can aid in this process.

Could you please let me know if my proposed solution is feasible with Azure services? If so, are there any examples or guides you could point me to that would help with implementing this approach?

*note: These are randomly selected publicly available financial statements.
Millns, Bailey 0 Reputation points

2023-07-19T18:31:00.05+00:00

Thank you for your response. My issue with table extraction is specifically related to inconsistencies in the layout of the same type of table across different financial documents. I am trying to extract the income table from UK financial statements, and I've found that the layout varies from one document to another.

Here are two examples for reference:

I did try training a custom extraction model as per your suggestion. However, I encountered errors due to incorrect labeling of the documents. At this stage, I am reluctant to spend more time labeling numerous documents for the model.

My preference is to use unsupervised extraction to pull all elements from the documents, and then leverage OpenAI to filter and identify the income table. I'm also open to using Azure Cognitive Search if it can aid in this process.

Could you please let me know if my proposed solution is feasible with Azure services? If so, are there any examples or guides you could point me to that would help with implementing this approach?

*note: These are randomly selected publicly available financial statements.
YutongTie-MSFT 48,586 Reputation points

2023-08-01T18:50:31.8+00:00

Hello @Millns, Bailey

Thanks for sharing the sample, so from your side you don't want to train the custom model from Form Recognizer side. We will forward it to product team to see if there any possible solutions.

My preference is to use unsupervised extraction to pull all elements from the documents, and then leverage OpenAI to filter and identify the income table.

If you extract all element from the original documents and leverage OpenAI to filter it, the final result can hardly be table again. Or you want to use OpenAI to make it as a table?

Share via

Combining Form Recognizer with OpenAI for Improved Table Extraction - Seeking Examples/Documentation