custom model train a table

Sai Ramesh 20

I am building a custom model to extract fields and line items from invoices. My table is dynamic, with fixed columns and rows spanning multiple pages. When training the model, do I need to enter the data for each row individually? For instance in a invoice, if I have 50 rows on pages one through five and 30 rows on page six, do I need to map a total of 280 rows of table data one row at a time?

Additionally, how many documents do I need to train for a particular invoice format? Is there a recommended number, such as 10 invoices per format? For example, if there are 5 different formats, should I train on 50 invoices? Table Mapping.png

dupammi 8,035 Reputation points Microsoft Vendor

2024-07-04T06:52:02.06+00:00
Hi @Sai Ramesh

Thank you for using the Microsoft Q&A forum.

For detailed guidance and best practices specific to table extraction and training custom models with Azure AI Document Intelligence, you may refer to their documentation here.

Mapping Rows in Tables:

When training a custom model to extract fields and line items from dynamic tables in invoices, meticulous labeling is key to ensuring accurate extraction across multiple pages and varying row counts. Given your scenario where tables span multiple pages and have varying row counts, you generally need to map each row individually during the training process. This ensures that the model learns to extract data correctly from each row, considering the dynamic nature of your tables.

Tips for Efficient Labeling:

Auto-Label Feature: You can use the auto-label feature for tables. If the auto-labeled layout extracts the result you need, use that result and skip the manual labeling process. Otherwise, start with creating the table field from the auto-labeled values and then edit as needed.

API Version: Use the latest API version for the best results. Ensure you label only the cells containing the data you need, which improves the model's focus and accuracy.

Training Document Count:

The number of documents required for training depends on the complexity and variability of the invoice formats. As a guideline, training on around 10 invoices per format is a good starting point. If you have 5 different formats, training on approximately 50 invoices in total would be beneficial. This helps the model learn the variations in layouts and content presentation across different invoices.

Additional Best Practices for Labeling:

Shift Select and Region Labeling: When labeling a large span of text, you can hold down the Shift key to select words quickly. For larger spans of text or when dealing with dense tables, consider using region labeling to select the entire area. The OCR results are populated in the value at training time, helping in efficient labeling.

Label Overlapping Fields: If your document has overlapping fields, label them by using the region labeling feature to select regions for each field. Overlapping fields are supported for both fields and table cells.

Field Subtypes: Choose the appropriate subtype when creating a field, such as selecting the dmy option for dates, to minimize post-processing and improve extraction accuracy.

I hope this helps.

Share via

custom model train a table