General document model key value pairs detection

Question

General document model key value pairs detection

Jacob12348771 0

Hi, I am currently completing a project in which Azure Document Intelligence is used as a part of an Azure Logic app, mapping the fields from a paper form to a record in Dataverse. However, I am having some issues with the consistency of the general document model and its detection of key-value pairs. Below are three examples demonstrating the problems I am having, with certain key-value pairs getting mapped incorrectly. 2024-01-19 15_18_33-Document Intelligence Studio - Microsoft Azure — Mozilla Firefox

2024-01-19 15_58_42-2024-01-19 15_15_39-Document Intelligence Studio - Microsoft Azure — Mozilla Fir

2024-01-19 15_33_09-Document Intelligence Studio - Microsoft Azure — Mozilla Firefox Is there any way I can ensure the consistency of the general document model for this? The forms uploaded will stay consistent. As you can see in the third image I attempted to increase the spacing between the two sections, however, this then resulted in some key-value pairs not even being recognised. This is using API Version: 2023-07-31 (3.1 General Availability). I have also tried the 'high resolution' add-on, along with switching the API version however the issues persist. I previously created a custom extraction model, however, this did not allow multiple documents to be in the same file otherwise values would only be populated once for one document. Any ideas what I can do to solve this?

santoshkc 15,600 Reputation points Microsoft External Staff Moderator

2024-01-22T09:34:11.2+00:00

Hi @Jacob12348771, Thank you for reaching out to Microsoft Q&A forum! As you are experiencing issues with the consistency of the general document model and its detection of key-value pairs. You have already tried several troubleshooting steps, including increasing the spacing between sections, etc. One thing you could try is to train the general document model with more examples of the forms you are using. This can help improve the accuracy of the model's key-value pair detection. Regarding custom model, you can try using a technique called "composed models." Composed models allow you to combine multiple custom models into a single model that can recognize and extract data from multiple document types. I hope this helps! Thank you.
santoshkc 15,600 Reputation points Microsoft External Staff Moderator

2024-01-23T08:36:30.0533333+00:00

Hi @Jacob12348771, We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Thank you!
Jacob12348771 0 Reputation points

2024-01-23T16:01:14.06+00:00

Thank you for your reply @santoshkc . Previously I created a custom extraction model which worked well for detecting fields on a single document. However, if there were multiple documents in a PDF each with the same form structure (for example the output from scanning several forms with a printer), the detection of fields would not work. Each field would only be recognised once with different fields being recognised on each document. This is shown below:

As you can see from the colour-coded detected fields, each field is only recognised once and never repeated, spread out across the analysed file. Therefore I can't see any benefit of using a composed model as my form will stay the same and will be using a model like I have already created. Is this a downside of using a custom model, only being able to process one form at a time? This example was analysing a pdf with 3 forms.
santoshkc 15,600 Reputation points Microsoft External Staff Moderator

2024-01-24T05:16:25.53+00:00

Hi @Jacob12348771, Thank you for providing more details about your issue and sorry for the trouble you are facing. To address this issue, you can try using a different approach to process your PDF files. Split the PDF file into individual documents and then process each document separately using your custom extraction model. You can use a PDF library or tool to split the PDF file into individual documents. I hope this helps! Thank you.

1 answer

Your answer

santoshkc 15,600 Reputation points Microsoft External Staff Moderator

2024-01-22T09:34:11.2+00:00

Hi @Jacob12348771, Thank you for reaching out to Microsoft Q&A forum! As you are experiencing issues with the consistency of the general document model and its detection of key-value pairs. You have already tried several troubleshooting steps, including increasing the spacing between sections, etc. One thing you could try is to train the general document model with more examples of the forms you are using. This can help improve the accuracy of the model's key-value pair detection. Regarding custom model, you can try using a technique called "composed models." Composed models allow you to combine multiple custom models into a single model that can recognize and extract data from multiple document types. I hope this helps! Thank you.
santoshkc 15,600 Reputation points Microsoft External Staff Moderator

2024-01-23T08:36:30.0533333+00:00

Hi @Jacob12348771, We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Thank you!
Jacob12348771 0 Reputation points

2024-01-23T16:01:14.06+00:00

Thank you for your reply @santoshkc . Previously I created a custom extraction model which worked well for detecting fields on a single document. However, if there were multiple documents in a PDF each with the same form structure (for example the output from scanning several forms with a printer), the detection of fields would not work. Each field would only be recognised once with different fields being recognised on each document. This is shown below:

As you can see from the colour-coded detected fields, each field is only recognised once and never repeated, spread out across the analysed file. Therefore I can't see any benefit of using a composed model as my form will stay the same and will be using a model like I have already created. Is this a downside of using a custom model, only being able to process one form at a time? This example was analysing a pdf with 3 forms.
santoshkc 15,600 Reputation points Microsoft External Staff Moderator

2024-01-24T05:16:25.53+00:00

Hi @Jacob12348771, Thank you for providing more details about your issue and sorry for the trouble you are facing. To address this issue, you can try using a different approach to process your PDF files. Split the PDF file into individual documents and then process each document separately using your custom extraction model. You can use a PDF library or tool to split the PDF file into individual documents. I hope this helps! Thank you.

Answer 1

Jacob12348771 0

Thank you @santoshkc , I am going to use a custom model. Initially I will send over the Base64 encoded string to an Azure Function to split the pages and then send back the Base64 for the individual pages and process the document contents for each of these.

YutongTie-MSFT 54,011 Reputation points Moderator

2024-02-07T20:56:23.1666667+00:00

@Jacob12348771 Thanks for sharing the workaround, please let us know if you have any questions again.

Share via

General document model key value pairs detection

1 answer

Your answer