Why the confidence score is different in Azure form recognizer and Microsoft Syntex?

Question

I am doing a comparison between the Microsoft Syntex and Azure Form Recognizer for IDP (Intelligent document processing) on the basis of performance and accuracy.

I have used same 15 training documents for both (Form Recognizer and Microsoft Syntex) to train a custom model. When I have tested both the technologies with same test document, output was same in Form Recognizer and Microsoft Syntex, but there was difference in the confidence score.

In more precise information, I have used custom template model in Azure Form recognizer and Custom Layout method in Microsoft Syntex.

Here, I want to know that why there is difference in the confidence score? How the confidence score is being calculated in the Azure From recognizer and Microsoft Syntex?

Thank you,

Answer 1

Hello @Subhash Kumar Mahato , Thanks for using Microsoft Q&A platform.

Firstly, coming to Azure Form Recognizer calculates confidence scores based on the probability that the prediction is correct. Confidence scores are estimated for predicted words, key-value pairs, selection marks, regions, and signatures. However, not all document fields return a confidence score.

As mentioned here in the documentation, Confidence scores generally have two data points. It's important to combine these two confidence scores (field level confidence score and the text extraction confidence score) to generate one overall score, which can help determine whether to automatically accept the prediction or flag it for human review, especially in scenarios where accuracy is critical.

I don't have much knowledge regarding the Microsoft Syntex. In Microsoft Syntex, the confidence score might be calculated using different techniques or methods. However, it's important to take these into consideration when evaluating the performance of both confidence scores.

I hope this helps.

Regards,
Vasavi

-Please kindly accept the answer and vote 'yes' if you feel helpful to support the community, thanks.