Search an image in a document using Azure Open AI using C#

Question

Hi Team,

I have a requirement where we can get the images from a source document. Let's say a document or a user manual that contains text and images it and needs to be used as source data for a chatbot.
For example: If the user wants to know the login process then the chatbot should be able to show the Login screen image from the source document in the response along with textual content.

Could you please advise if we can implement something using Azure Open AI with C#?

Regards,

Mohit

Answer 1

Hi @Madan, Mohit

Welcome to Microsoft Q&A! Thanks for posting the question.

Here is something you can do to implement your solution -

• To extract images from a source document and use them in a chatbot response, you can use optical character recognition (OCR) and image processing techniques to identify and extract the relevant images from the document. You can then store these images in Azure Blob Storage and extract the text from these images to add to your Azure Search Index. Please refer to the OCR - Optical Character Recognition documentation for details.
• To process the text content of the document using the Text Analytics API, you can use the key phrase extraction and entity recognition features to identify relevant keywords and concepts. You then need to add your summarized text to your Azure Search Index. Please refer to documentation for details.

As Azure search can be integrated with Azure Open AI, you then use the Azure Search service with the Open AI with your own data to include int the chatbot response.

You can also refer to this solution for extracting images and using text api for reference.

Hope this helps.

Thanks

Saurabh

---

Please 'Accept as answer' and Upvote if it helped so that it can help others in the community looking for help on similar topics.