Any Azure AI Service to check for inconsistencies of contents within a document?

Question

Hello, I am looking for an Azure AI service that can detect contradictions within a document, such as a PDF, CSV, or other formats. I have tried extracting each sentence, creating sentence pairs, and using OpenAI to identify the contradictions. However, this is not feasible for my document because it has over 8,000 sentences, which means there are about 31,000,000 pairs of sentences to check. Is there a service that I can upload the entire document to and receive a list of contradictions? Alternatively, are there any workarounds available?
Thank you for your assistance.

Answer 1

Hi @PHAM Thanhnhan (HELLA)

Welcome to Microsoft Q&A! Thanks for posting the question.

As per my understanding of your problem, I don't think we have any Azure AI service specifically available to collect documents to identify any contradictions.

However, you may look into the below options which you can explore if this to get the expected results -

• Train an Azure Machine Learning model to identify contradictions in your documents. You can use a supervised learning approach where you need to label your data with examples of contradictions and non-contradictions. Then you can use this label then you can use this labeled data to train a classification model and use the same model for your purpose. But this process can be a complex and time-consuming process and you will need a good understanding of ML concepts and techniques.
  Ref: Tutorial: Train a classification model with no-code AutoML in the Azure Machine Learning studio
• Explore conversational language understanding (CLU) service which can help you to identify the intent and meaning behind the text. You can use custom classification feature and train using specific phrases or patterns that may indicate a contradiction.

Hope this helps.

Thanks

Saurabh

Please 'Accept as answer' and Upvote if it helped so that it can help others in the community looking for help on similar topics.