DI custom extractor feature problem

Question

Hey everyone,

I'm struggling with table extraction in my custom Document Intelligence model. It keeps identifying what is clearly one single table as two separate ones. This is causing problems for my downstream processes.

I've already tried training it by:

Labeling the entire table area as a "field."

Labeling the entire table area specifically as a "table."

Neither approach worked; the model still splits the table after training.

My main question is: How can I get the model to recognize the entire table as one single thing?

I'm starting to suspect the underlying pre-trained model is the real issue here and my custom model is just inheriting its behavior. Has anyone run into this or found a workaround?

Any suggestions would be great. Thanks!

Answer 1

Hello Arthur Reis,

Welcome to Microsoft Q&A & Thank you for reaching out to us.

I understand how frustrating it can be when a single table is incorrectly split into multiple segments by your custom Document Intelligence model especially when this impacts downstream processing.

Here are some steps you could try to resolve the issue:

The table detection logic in a custom extractor model is heavily influenced by the underlying pre-trained model. If there are visual gaps, uneven alignments, extra white spaces, or separators within the table structure, the model may interpret them as separate tables even if they appear as one continuous table visually. This is a known behavior, particularly in complex or multi-section layouts.

First, review the structure of your document closely. Ensure there are no visual or structural elements such as extra horizontal or vertical lines, large whitespace gaps, or unrelated annotations that could cause the model to misinterpret a single table as multiple ones. Even small layout inconsistencies can confuse the model and lead to table fragmentation.

Since you’ve already tried labeling the table in different ways, refining the training data is essential. Here are some best practices to follow:

• Label the entire table region as one single table field instead of labeling individual columns or rows.

Ensure the bounding box tightly covers all rows and columns without leaving extra margins or whitespace.

Provide diverse training examples of the same table type, including variations in row counts, spacing, and formatting.

If the table spans multiple pages, include such examples during training so the model learns how to interpret them correctly.

To further improve model accuracy, consider the following approaches:

Train the model with a larger variety of table samples so it can better understand what a complete table looks like.

If your tables span multiple pages, include samples that reflect this to teach the model how to treat them as one unit.

Review and adjust any available table extraction parameters in the extractor settings. Some options allow you to define how the model should treat closely located table fragments, merging them into one.

If supported, you can refine your custom extractor to better interpret complex layouts:

Use extractor refinement options to specify that multiple detected fragments should be treated as a single table.

Create structured templates that define what a single, complete table should look like. This can significantly improve the model’s recognition accuracy.

Inspect the document’s formatting and consider preprocessing steps such as normalizing whitespace, removing decorative lines, or aligning table elements before extraction. If preprocessing isn’t feasible, consider post-processing the model’s output to merge detected tables if they share the same schema or structure.

If basic adjustments don’t resolve the issue, more advanced options are available:

Segment your document into logical sections before applying table extraction.

Fine-tune how table fragments are interpreted and combined.

Extract raw table structures and apply custom logic downstream to merge multiple detected tables into one.

Please refer this Build and train a custom extraction model, Document Intelligence custom models.

I Hope this helps. Do let me know if you have any further queries.

Thank you!