Share via

Understanding the workings of the Azure OCR Custom Model.

Qadir Varawala 11 Reputation points
2021-08-18T08:27:21.317+00:00

Things we tried.
We currently train some sample files with multiple tables and key Value Pairs on the PDF.

1) when we train the model well, with 100% accuracy, and the parse a new file which hasnt been trained before, we get have low confidence rates on the data from the new file. Some attributes arent even mapped.

2) When We have similar format files with different data, the accuracy of the model isnt 100%. We get better confidence rates, but still not in the 90's like we expect. The confidence rates are between 20 -80 % .

3) We then believed the way we were tagging the data was a deterrent to getting better confidence scores. We renamed he tags , as table1, table2 etc hoping to get better accuracy while reading the data from the model . We still rceived lower confidence scores.

There are a few points I'd like to understand .

1) When we tag the data, is the model AI going to base future scans on the position of the data in the PDF, or the actuall content itself.

For example, if I tag the word 'Microsoft' in a document... will it look for the word 'Microsoft in all pages of the PDF, or will it look for it in the location based on the files that were trained.

2) What can we do to improve the confidence score of files being scanned in the future ?

3) The Tables we have, are not always in the same coordinates, they would vary based on the data that we have. We found it dificult selecting the entire table, as a couple of columns would get skipped. We tagged each of the values from the table. In case there are more rows than what I have tagged in the model. will I be able to get the data from those additional rows.

Azure AI Document Intelligence
Azure AI Document Intelligence
An Azure service that turns documents into usable data. Previously known as Azure Form Recognizer.
1,447 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Qadir Varawala 11 Reputation points
    2021-08-18T12:56:54.14+00:00

    Hi Ram ,

    This is the too we are currently using . https://fott.azurewebsites.net/#

    What is the latest version of the Tool., because I dont seem to be able to map separate sections of the page as different models

    1 person found this answer helpful.
  2. Ramr-msft 17,621 Reputation points
    2021-08-18T09:58:56.977+00:00

    @Qadir Varawala Thanks for the question. Can you please share the sample documents and FR version that you are trying. Generally we start with 5 documents as training set and you should be able to add more documents to your training incrementally to see an improvement in the results. If you don’t see improvements after doing that, We can forward to the Forms Recognizer team.
    There are few ways you can do it here:
    • Create Model for each segment on the page. So if you have 3 segments you would create 3 models and call individual models.
    • Same as above but create a compose model. The compose model will automatically classify your document and provide the data for the respective tag/data you’re looking for.
    • Create one model for and capture datapoints for each segment in a single or set of tags. After the data is extracted, you can use a function for post processing and break down the values by segments.

    Please follow the document to improve the results.

    https://github.com/microsoft/knowledge-extraction-recipes-forms