Filter data based on metadata

Question

Filter data based on metadata

Alessandro Sorcinelli 0

Hi,
I'm developing an application based on Azure OpenAI. In particular, the application should provide the user with the possibility to search, via ChatGPT, private documents stored on a Blob Storage.

Documents are organized into categories and the system must allow you to filter the documents by searching only within a certain categories. It seems to me that the only ways to filter data are Security Filters and Content Filters, but both types do not fit this scenario.

Is it possible, from ChatGPT, to impose a filter based on metadata? I use Azure Cognitive Search to index content

Thanks.

Ramr-msft 17,821 Reputation points

2023-11-27T12:21:49.5133333+00:00

@Alessandro Sorcinelli Thanks for the question, Have a look at Filtering with metadata (pinecone.io). Document level access control is something you must build in your application layer, together with the right fields in your vector db. See Security filters for trimming results - Azure Cognitive Search | Microsoft Learn to understand how this would work with Cognitive Search.

In short, when your user is logged in, you will retrieve their group id or user id. When your user asks a question, you will do the query to Pinecone in the background and add a search filter to only query the documents the user has access to.

Eventually this kind of feature could be added to the "Chat with your Data" feature in Azure AI Studio, but for now this is not there yet.

Your answer

Ramr-msft 17,821 Reputation points

2023-11-27T12:21:49.5133333+00:00

@Alessandro Sorcinelli Thanks for the question, Have a look at Filtering with metadata (pinecone.io). Document level access control is something you must build in your application layer, together with the right fields in your vector db. See Security filters for trimming results - Azure Cognitive Search | Microsoft Learn to understand how this would work with Cognitive Search.

In short, when your user is logged in, you will retrieve their group id or user id. When your user asks a question, you will do the query to Pinecone in the background and add a search filter to only query the documents the user has access to.

Eventually this kind of feature could be added to the "Chat with your Data" feature in Azure AI Studio, but for now this is not there yet.

Share via

Filter data based on metadata

Your answer