How to make Azure AI search return images (in OpenAI service - from your own data)

Question

Hello, I am working on a RAG application with Azure AI search to index my PDF documents stored in Azure blob storage. The documents contains figures. The end goal is to create azure openai service (from your own data) to have a chatbot on my documents. The documents are 1000 pages long and contain figures.

I want :

1. that the answers of azure OpenAI service contain the related figures from the indexed document when needed
2. Do a variable size chunking based on the markedownformat since my documents are so long.

Answer 1

The original PDF documents are not store directly in Azure AI Search. It will be processed and chunked during the ingestion phases.

1- the PDF document (text content) will be broken down into smaller chunks and stored in AI Search as individual records. then embedding will be created for text for vector search

https://learn.microsoft.com/en-us/azure/architecture/solution-ideas/articles/ai-search-skillsets

2- If your requirements is to allow user to search figures by keywords: As the figures/diagram is not imported with the text from the PDF, you will need to do additional work to make these figures/diagram stored and searchable in the AI Search. i.e. generate image embedding for the figures (use Vision & embedding model) and enable them for vector search.
3- If your requirements is to just display the figures together with returned text: you probably need to store the figures/images separately in a storage account and find a way to reference them inside text documents and display in the application.