OCR ignores numbers in the corner of the document

Vlad R 20

Hello! I am using the Read model to extract text information from receipts. It seems to ignore numbers that are located in the bottom corner of a document. I am attaching three examples where the Read model has not read the 000 code at the bottom of the receipt.

I tried adding another number to the left of the code manually (to make seem it like "5000", or "1000", etc.), but it still just ignores the code completely.

Can anything be done about that by my side (e.g. image preprocessing, rotating the image, etc.?)

test5.jpg
test4.jpg
test2.jpg

VasaviLankipalle-MSFT 14,576 Reputation points

2023-03-23T22:00:38.7166667+00:00

Hi @Vlad R , Thanks for using Microsoft Q&A Platform.

It is possible that the Read model is not recognizing the numbers in the bottom corner of the receipt due to the quality of the image. For best results, you can try providing one clear photo or high-quality scan per document. You can also try rotating the image to see if that helps the model recognize the numbers.

I tried resizing the image and it worked for me. I would advise you to provide a high-quality image and test it again.

Additionally, you can try using a different pre-built model or training your own custom model to better recognize the numbers in the bottom corner of the receipt.

I hope this helps.

Regards,
Vasavi
Vlad R 20 Reputation points

2023-03-24T11:58:29.4333333+00:00

Vasavi, thanks for the quick response!

Unfortunately, image rotation does not help. I have also tried to resize the images (make them x1.5, x2, x4 times smaller), but that did not help either.

We are using the API to load the images of receipts automatically, and the receipts have different sizes. So it is not possible for us to crop the pictures as you did in your example.

We are getting the photos from our users' devices, so we can not expect the photos to have exceptional quality. The other text fields that we are extracting from the same receipts are read almost perfectly (and are at least detected by the OCR model). So it is probably not the image quality?

I have more examples with original (not preprocessed, and not redacted) images, but I do not want to share them in a public forum. Is there another way to send those?
VasaviLankipalle-MSFT 14,576 Reputation points

2023-03-24T21:16:23.39+00:00

Hi @Vlad R , Thank you for your detailed information.

If the data in non-confidential, could you please send an email to azcommunity@microsoft.com with the below details, so that we can reproduce the same on our end.

Subject: Attn: Vasavi
Subscription ID:
Thread URL: Link to this thread/question
Sample documents

Please let me know here once you have done the same.
Vlad R 20 Reputation points

2023-03-27T09:37:50.79+00:00

@VasaviLankipalle-MSFT ,

I have sent the email with further info.
VasaviLankipalle-MSFT 14,576 Reputation points

2023-03-28T00:49:37.8566667+00:00

@Vlad R , thanks for sharing. Please allow some time to check internally on this.

Accepted answer

VasaviLankipalle-MSFT 14,576 Reputation points

2023-03-29T18:19:47.5033333+00:00

Hi @Vlad R , Thank you for your patience.

The current system has a limitation in text detection and rejection. So even if the text contains characters from unknown languages, it can still be detected. However, the text recognizer cannot recognize these unknown characters, resulting in the entire text line being rejected. This can lead to numbers/Latin characters being detected along with the unknown text, causing the recognizer to fail to recognize the unknown characters, and ultimately deleting the entire line, including the recognized characters.

After checking with the PG team, noticed that this is a known issue. Sorry for the inconveniences.
Maybe you can try custom models and see if that helps.

I hope this helps. Let me know if you need more information.

Regards,
Vasavi

-Please kindly accept the answer and vote 'Yes' if you feel helpful to support the community, thanks.
Please sign in to rate this answer.

1 person found this answer helpful.

0 comments No comments
Sign in to comment

OCR ignores numbers in the corner of the document

0 additional answers