OCR Character Recognition Accuracy: When analyzing the document (see the inserted example), certain letters, such as "Y," are being misinterpreted as "<" or other incorrect characters. This creates inconsistencies in the extracted data. Some text elements in the document are not detected at all, even though they are visible and clear. Would focusing the training process on individual table columns improve detection accuracy? What are the recommended approaches for training and testing models on structured documents like financial reports? Are there specific criteria or critical steps to ensure the model generalizes well across similar documents? Labeling Workflow Limitations: During the labeling process, the platform does not allow me to manually modify incorrect characters or text detected by the OCR engine. Even when I download the labels.json file, make corrections manually, and re-upload it, the changes are not reflected in the system. This hinders the ability to refine the training data effectively. Is there a straightforward method to modify OCR-detected text during the labeling step? UI Functionality Missing for Comparison: I could not find an option in the UI to visually compare the raw image with the OCR-processed version side by side. Such a feature would make it easier to identify recognition errors and validate corrections. Does Azure Document Intelligence provide a feature to visually compare the raw image with the OCR'd result in the same interface? If not, are there any suggested workflows or tools for doing this?

Issues with Azure Document Intelligence OCR and Labeling

jonasmil 10

OCR Character Recognition Accuracy:
- When analyzing the document (see the inserted example), certain letters, such as "Y," are being misinterpreted as "<" or other incorrect characters. This creates inconsistencies in the extracted data.
- Some text elements in the document are not detected at all, even though they are visible and clear.
- Would focusing the training process on individual table columns improve detection accuracy?
- What are the recommended approaches for training and testing models on structured documents like financial reports?
- Are there specific criteria or critical steps to ensure the model generalizes well across similar documents?
Labeling Workflow Limitations:
- During the labeling process, the platform does not allow me to manually modify incorrect characters or text detected by the OCR engine.
- Even when I download the labels.json file, make corrections manually, and re-upload it, the changes are not reflected in the system. This hinders the ability to refine the training data effectively.
- Is there a straightforward method to modify OCR-detected text during the labeling step?
UI Functionality Missing for Comparison:
- I could not find an option in the UI to visually compare the raw image with the OCR-processed version side by side. Such a feature would make it easier to identify recognition errors and validate corrections.
- Does Azure Document Intelligence provide a feature to visually compare the raw image with the OCR'd result in the same interface? If not, are there any suggested workflows or tools for doing this?

romungi-MSFT 47,441 Reputation points Microsoft Employee

2024-11-27T08:59:59.3533333+00:00

@jonasmil I think the recommendation with respect to recognition is usually to train the model with more documents if the accuracy is low. Basically, a balanced dataset is recommended for different project types i.e. template or neural model types. In this case since the letter Y is not recognized correctly in this image, you can try to label it or try to increase the font size if this is a PDF type of document and retry the OCR process.

The training process depends on the output of OCR process and the labeling done in the document. Generally, you label fields that you plan to use as labels. So, start by identifying the fields and start the labeling process.

With respect to the limitations you mentioned, updating the extracted OCR and labels JSON files is not recommended as it may corrupt the project or model training. Current API versions do not have the capability to perform modification of the current OCR output.

With respect to the UI functionality, there is no direct way to compare the raw image and OCR image side by side. But you can open two instances of your studio in the browser and compare them. But again, the yellow highlighted part of the image is what is identified by OCR process so you can conclude that the non-highlighted part of an image is what was missed in OCR process.

I hope this helps!!
romungi-MSFT 47,441 Reputation points Microsoft Employee

2024-11-29T08:40:36.64+00:00

@jonasmil Did you get a chance to check if the above response helped? Thanks!!
jonasmil 10 Reputation points

2024-12-09T10:25:48.28+00:00
Actually I have still problems, I would be glad if detailed answer for each following questions:

Our main problem is the Azure OCR is not detecting at all some single characters even they are preprocessed. We tried extracted columns to improve results but it didn't work also. How come other open source OCR project could detect it but not Azure, what may be reasons for this (could it be characters close to the line, is it minimum pixel per box size related etc.), extracted columns works poorly than all page processed, how to overcome this ? This is very critical for our progress, because if OCR cannot detect characters, we cannot label it and we cannot use it at all.

How model calculates confidence values, what is the logic behind it ?

How can we learn some statistics that we used by Azure, like column count, character count ? Can we compare training data and performance data, ( e.g column-wise performance, page-wise performance, can we create confusion matrix ?

single_column.jpg

Share via

Issues with Azure Document Intelligence OCR and Labeling

Your answer