Filtering data during retrieval using Azure OpenAI portal

Question

Filtering data during retrieval using Azure OpenAI portal

Adit Khandelwal 0

I am trying to create a webapp using Azure OpenAI by connecting my own data which is vectorized using Azure AI search. I have added some custom fields and I want the data to get filtered based on those custom tag and user's query during retrieval. For example if there is a category tag with values like technology, marketing, etc. I want data to be filtered based on this tag and user's query and then retrieve most relevant documents from the filtered data. Can this be achieved using no code way provided in Azure OpenAI Portal where we can deploy as webapp after connecting to our data. If not what is the best and easiest way to set this up (either via code or no code approach)?

Ravada Shivaprasad 535 Reputation points Microsoft External Staff Moderator

2025-05-07T19:39:41.7+00:00

Hi Adit Khandelwal

To implement custom field-based filtering with Azure OpenAI and Azure Cognitive Search, Microsoft recommends a code-based solution rather than relying on the Azure OpenAI Portal, which lacks advanced customization capabilities. This approach ensures greater flexibility for filtering and retrieving relevant documents. The architecture integrates key components, including filter logic to process user queries and selected categories, custom fields storage to manage category tags and metadata, Azure Cognitive Search to perform retrieval based on filtering criteria and the original query, and finally, the Azure OpenAI model, which generates responses based on retrieved content.For implementation, developers can use Python with Flask or Node.js with Express to build a RESTful API that handles both filtering and query processing efficiently. The Python Flask approach provides a structured separation between search and AI services, ensuring scalability and maintainability. Meanwhile, the Node.js Express implementation leverages asynchronous operations for better performance in web applications. Both options integrate directly with Azure Cognitive Search for filtering and Azure OpenAI for response generation. This ensures that responses are based only on relevant documents, minimizing unnecessary data retrieval and improving accuracy.

Deployment involves setting up Azure OpenAI Service, Azure Cognitive Search, and an App Service to host the application. Environment variables such as SEARCH_ENDPOINT, SEARCH_KEY, OPENAI_ENDPOINT, and OPENAI_KEY must be properly configured. Once the application is developed, pushing the code to Azure DevOps or GitHub and setting up continuous deployment to Azure App Service is recommended for efficient management. Additionally, configuring CORS settings ensures secure and controlled access.

Security considerations are critical when implementing this solution. Network configurations should restrict access to specific networks while enforcing authentication mechanisms. Using Azure Key Vault for credential storage is essential to prevent unauthorized access. Data protection strategies, such as encryption at rest, rate limiting, and usage monitoring, help maintain security and reliability. These measures provide enhanced data privacy and compliance while optimizing filtering processes.

While this approach requires more setup than a portal-only solution, it offers the necessary customization for field-based filtering and ensures optimal integration between Azure OpenAI and Azure Cognitive Search. This framework is ideal for organizations needing precise search and AI-based document processing.

Reference - Content filtering, Use a blocklist with Azure OpenAI ,- GitHub Repository for Azure AI Documentation

Thanks
Adit Khandelwal 0 Reputation points

2025-05-08T04:31:40.0566667+00:00

Is there a github repo with python codebase which provides the base framework for chatbot implementation where I can update the content filtering and OpenAI implementation as per my need keeping other components like authentication, chat history, etc. as is ? I want this to be deployed and used in MS teams. I have seen some repos like https://github.com/microsoft/sample-app-aoai-chatGPT but wanted to find out the best one available.

1 answer

Your answer

Adit Khandelwal 0 Reputation points

2025-05-08T04:31:40.0566667+00:00

Is there a github repo with python codebase which provides the base framework for chatbot implementation where I can update the content filtering and OpenAI implementation as per my need keeping other components like authentication, chat history, etc. as is ? I want this to be deployed and used in MS teams. I have seen some repos like https://github.com/microsoft/sample-app-aoai-chatGPT but wanted to find out the best one available.

Answer 1

Hello @Adit Khandelwal ,

First thing you need check is cases where the chat completion gives you response properly.

As per this documentation you can see.

The chat models perform best in question and answer exercises. For example, "What are my available health plans?" or "What is the health plus option?".Queries that require data analysis would probably fail, such as "Which health plan is most popular?".Queries that require information about all of your data will also likely fail, such as "How many documents have I uploaded?". Remember that the search engine looks for chunks having exact or similar terms, phrases, or construction to the query. And while the model might understand the question, if search results are chunks from the data set, it's not the right information to answer that kind of question.

In your case the requirement is filter the documents in AI search based on the tags, for that you should be having the field to store tag values in your search index and make it Filterable.

User's image

Above I am having field Rating and made filterable, similarly you create tags field.

Below is the sample query for hotel data.

User's image

Here, i have also added system message in the left side, same kind of message you should add on your data according to your requirement.

Next, we have a deploy option which directly create a web app for you with added data source.

User's image

Please try above and if find any query do let us know in comments or private message.

Thank you

JAYA SHANKAR G S 4,035 Reputation points Microsoft External Staff Moderator

2025-05-19T06:43:04.68+00:00

Hello @Adit Khandelwal ,

Did the above answer helped you? please check and let us know if you have any query.

Thank you
JAYA SHANKAR G S 4,035 Reputation points Microsoft External Staff Moderator

2025-05-20T05:22:36.14+00:00

Hello @Adit Khandelwal ,

I believe the given above answer helped you, if you have any query do let us know in comments.

Thank you

Share via

Filtering data during retrieval using Azure OpenAI portal

1 answer

Your answer