Azure OpenAI responses based on documents in SharePoint

Question

Azure OpenAI responses based on documents in SharePoint

Jamil Hallal 21

My data is a list of documents in SharePoint. I'm using one index in coginitive search to index all documents. The document library in SharePoint contains all documents uploaded by different users. how can I configure azure openai to only return responses from documents uploaded by a the user himself? Which means a can user can analyze or summarize his own documents only with no access to documents uploaded by someone else. I tried to add filter to the cognitive search in the datasources passed to openai but it didn't work, here is the code of the request that didn't work:

{
  "dataSources": [
    {
      "type": "AzureCognitiveSearch",
      "parameters": {
        "endpoint": "https://mycognitivesearch-openai.search.windows.net/indexes/myindex/docs?api-version=2023-07-01-Preview&search=*&filter=search.ismatch(%27abc title%27%2C%20%27title%27)",
        "key": "mykey",
        "indexName": "myindex",
        "roleInformation": "You are an AI assistant that helps people find information."
      }
    }
  ],
  "messages": [
    {
      "role": "user",
      "content": "what is machine learning?"
    }
  ]
}

In the previous syntax, I tried to filter the data returned by the cognitive search to make sure OpenAI will analyze only chunks with title 'abc title' but it didn't work, i was still able to ask and receive answers of documents uploaded by someone else which should not be the case. Is there a way to add filter syntax to the cognitive search when passing the dataSources to Azure OpenAI?

romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2023-08-07T13:21:37.4533333+00:00

@Jamil Hallal I think you are setting the filter at the endpoint URI to use a particular document from the index. The dataSources configuration in this case will only take the endpoint URI and use the index to fetch the data. In this scenario, creating an index for each user might be the only option to fetch data related to the user.
Jamil Hallal 21 Reputation points

2023-08-07T13:33:50.4366667+00:00

@romungi-MSFT But there is a limitation on the number of indexes we can create (of course depending on the subscription plan) and the higher plan is very expensive. Plus if I create an index per user then I will have to manage the indexes when users join or leave the company.
Anjan Shakya 100 Reputation points

2023-11-11T13:45:03.7233333+00:00

I also have same issue, I have multiple pdfs and they are all indexed in one index.
Now I need to make the open ai response based on specific document only.
This is the use case when my app user wants to ask questions related to a specific document only.

Is there a way we can do it by adding some filter parameter?

Your answer

romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2023-08-07T13:21:37.4533333+00:00

@Jamil Hallal I think you are setting the filter at the endpoint URI to use a particular document from the index. The dataSources configuration in this case will only take the endpoint URI and use the index to fetch the data. In this scenario, creating an index for each user might be the only option to fetch data related to the user.
Jamil Hallal 21 Reputation points

2023-08-07T13:33:50.4366667+00:00

@romungi-MSFT But there is a limitation on the number of indexes we can create (of course depending on the subscription plan) and the higher plan is very expensive. Plus if I create an index per user then I will have to manage the indexes when users join or leave the company.
Anjan Shakya 100 Reputation points

2023-11-11T13:45:03.7233333+00:00

I also have same issue, I have multiple pdfs and they are all indexed in one index.
Now I need to make the open ai response based on specific document only.
This is the use case when my app user wants to ask questions related to a specific document only.

Is there a way we can do it by adding some filter parameter?

Share via

Azure OpenAI responses based on documents in SharePoint

Your answer