Manually Fix Word Boundaries after OCR ?

Question

Manually Fix Word Boundaries after OCR ?

Cyril Carraz 31

Context :

Using Form Recognizer v3
Training a Custom Neural Model

Problem :

After analysing a file with handwritten texts, sometimes the OCR bundles a lot of words together in the same clickable entity, which prohibits me from labeling the fields correctly.

Example :

User's image

Here, I should have a field called 'Version' that grabs 'v25.1' or at least 'v25.1)', but I feel like labeling the field as 'v25.1)/outcome :' would invite errors later on.

Sometimes, it's worse because the OCRed entity represents two separate fields and I'm forced to choose which one to label between them.

Question :

Is there a way to manually fix the word boundaries ? Maybe through the json files generated ?

VasaviLankipalle-MSFT 18,676 Reputation points Moderator

2023-03-02T22:50:59.4266667+00:00

Hi @Cyril Carraz , Thanks for using Microsoft Q&A Platform.

Have you tried labeling the fields with the Shift select method?

I tried the shift select method and was able to reproduce your sample and extract the version value. So, this is one method you can try and let us know if that helps in your case.

Here are the some of the best practices for custom labeling, please try to check this: https://learn.microsoft.com/en-us/azure/applied-ai-services/form-recognizer/concept-custom-label-tips?view=form-recog-3.0.0

Regards,
Vasavi
Cyril Carraz 31 Reputation points

2023-03-03T08:38:05.7066667+00:00
I tried it and it didn't work, it selects the same thing. I imagine it worked for you because the zoom level isn't the same ?

Here's another example :

Here, the "words" detected by the OCR are :

(1001

'4254v25.1)/

Whereas I want them to be at least like :

(10014254

v25.1)

Or even better (but the former result is easily converted to this in the post-processing step) :

(

10014254

v25.1

)

Kind Regards,
VasaviLankipalle-MSFT 18,676 Reputation points Moderator

2023-03-04T01:07:11.8733333+00:00

Hi @Cyril Carraz , I can understand your situation. I believe since the handwritten text is not that clear might also be the reason for not recognizing well. As we know the quality of the text also impacts the accuracy of the model: https://learn.microsoft.com/en-us/azure/applied-ai-services/form-recognizer/concept-custom?view=form-recog-3.0.0#custom-model-development-options

If there is a space between words or characters, it will automatically recognize as two words. Also, I'm checking internally regarding modifying the boundary region values in the file.labels.json(Azure->Blob storage). Will get back to you on this.

Meanwhile, if you don't have any confidential information, you can share the document page you're discussing with us. So that we can reproduce and get back to you.
HouMez 391 Reputation points

2023-03-08T13:51:46.3533333+00:00

Hi @VasaviLankipalle-MSFT,

Thank you for you suggestions.
I tried updating the .ocr.json files, I was able to pick the proper text while labeling my documents as you can see in the example below;

Before updating the .ocr.json file:

After updating the .ocr.json file:

But when I run the model (tested with Neural and Template builds) I get the old/original ocr text.
VasaviLankipalle-MSFT 18,676 Reputation points Moderator

2023-03-08T21:28:59.7366667+00:00

Hi HouMez, Thanks for sharing this with us. As I mentioned earlier, adjusting ocr.json files and training the Custom neural model will not help because the region is only used to map OCR words the model still takes OCR words. That's the reason you won't see any difference in the result.
VasaviLankipalle-MSFT 18,676 Reputation points Moderator

2023-03-16T04:58:25.54+00:00

@Cyril Carraz , did you get a chance to check my response? Please check and take time to accept the answer if you feel helpful to support the community.

1 answer

Your answer

VasaviLankipalle-MSFT 18,676 Reputation points Moderator

2023-03-02T22:50:59.4266667+00:00

Hi @Cyril Carraz , Thanks for using Microsoft Q&A Platform.

Have you tried labeling the fields with the Shift select method?

I tried the shift select method and was able to reproduce your sample and extract the version value. So, this is one method you can try and let us know if that helps in your case.

Here are the some of the best practices for custom labeling, please try to check this: https://learn.microsoft.com/en-us/azure/applied-ai-services/form-recognizer/concept-custom-label-tips?view=form-recog-3.0.0

Regards,
Vasavi
Cyril Carraz 31 Reputation points

2023-03-03T08:38:05.7066667+00:00

I tried it and it didn't work, it selects the same thing. I imagine it worked for you because the zoom level isn't the same ?

Here's another example :

Here, the "words" detected by the OCR are :

(1001

'4254v25.1)/

Whereas I want them to be at least like :

(10014254

v25.1)

Or even better (but the former result is easily converted to this in the post-processing step) :

(

10014254

v25.1

)

Kind Regards,
VasaviLankipalle-MSFT 18,676 Reputation points Moderator

2023-03-04T01:07:11.8733333+00:00

Hi @Cyril Carraz , I can understand your situation. I believe since the handwritten text is not that clear might also be the reason for not recognizing well. As we know the quality of the text also impacts the accuracy of the model: https://learn.microsoft.com/en-us/azure/applied-ai-services/form-recognizer/concept-custom?view=form-recog-3.0.0#custom-model-development-options

If there is a space between words or characters, it will automatically recognize as two words. Also, I'm checking internally regarding modifying the boundary region values in the file.labels.json(Azure->Blob storage). Will get back to you on this.

Meanwhile, if you don't have any confidential information, you can share the document page you're discussing with us. So that we can reproduce and get back to you.
HouMez 391 Reputation points

2023-03-08T13:51:46.3533333+00:00

Hi @VasaviLankipalle-MSFT,

Thank you for you suggestions.
I tried updating the .ocr.json files, I was able to pick the proper text while labeling my documents as you can see in the example below;

Before updating the .ocr.json file:

After updating the .ocr.json file:

But when I run the model (tested with Neural and Template builds) I get the old/original ocr text.
VasaviLankipalle-MSFT 18,676 Reputation points Moderator

2023-03-08T21:28:59.7366667+00:00

Hi HouMez, Thanks for sharing this with us. As I mentioned earlier, adjusting ocr.json files and training the Custom neural model will not help because the region is only used to map OCR words the model still takes OCR words. That's the reason you won't see any difference in the result.
VasaviLankipalle-MSFT 18,676 Reputation points Moderator

2023-03-16T04:58:25.54+00:00

@Cyril Carraz , did you get a chance to check my response? Please check and take time to accept the answer if you feel helpful to support the community.

Answer 1

Hi @Cyril Carraz , in Custom Neural Model, manually adjusting the boundary will not help because the region is only used to map OCR words, the model still takes OCR words as input in both training and analyzing, and if OCR doesn't split words correctly, the result may still contain extra characters or be missing some characters.

If you still want to correct the bounding box for the Custom Neural Model, then you must first update the ocr.json (Azure portal->Blob storage) file, after which you will be able to select the corrected words.

Additionally, Region labeling can also be used to correct the region for Custom Template Model.
I have already shared this feedback to PG team.

I hope it helps.

--Please kindly accept the answer and vote 'Yes' if you feel helpful to support the community, thanks.

Regards,
Vasavi

Share via

Manually Fix Word Boundaries after OCR ?

1 answer

Your answer