How does the layout model determine reading order?

Bogdan Pechounov 60

Initially when I was using the AzureAIDocumentIntelligenceLoader with Langchain, I thought that the API responds with an array of paragraphs and that the markdown is assembled on the client. Is the reading order determined on the server (closed source); can the algorithm be tweaked?

Are there any guarantees that the order calculated is always correct?

YutongTie-MSFT 50,856 Reputation points

2024-03-06T01:04:51.0433333+00:00

@Bogdan Pechounov Thanks for reaching out to us, we are checking on this issue internally, will let you know later.
Bogdan Pechounov 60 Reputation points

2024-03-14T17:07:18.89+00:00

@YutongTie-MSFT Thank you. I found an article on LayoutReader, something similar might be used.
YutongTie-MSFT 50,856 Reputation points

2024-03-14T17:26:25.3933333+00:00

@Bogdan Pechounov Thanks for sharing the solution! Please let us know if you have any further questions.

Regards,

Yutong
Bogdan Pechounov 60 Reputation points

2024-03-14T17:37:02.83+00:00

@YutongTie-MSFT (It was supposed to be a comment, not sure what happened). Is it correct that the LayoutReader (relying on LayoutLM), or a higher version of it (working with LayoutLMv3?), is being used to determine reading order? (so it's not an algorithm that can be customized)

I am wandering if there are any results, metrics or benchmarks to have a better idea of how well the model determines reading order.
YutongTie-MSFT 50,856 Reputation points

2024-03-14T17:40:41.95+00:00

@Bogdan Pechounov Yes, mine is comment as well. I am checking with internal about those details you need.

Regards,

Yutong

Share via

How does the layout model determine reading order?

Your answer