Azure AI Search
An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.
Issues on RAG application over large data
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 77%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3E0%3C/text%3E%3C/svg%3E)
08382095
65
Reputation points
Hi,
We have built one of the RAG applications where the source data is in SharePoint and have used Azure AI search, Azure OpenAI and Azure storage.
We have populated azure container with all the files(around 1100) in a sharepoint site from multiple document libraries through connectors in the logic apps and created an AI search indexer using AI search API endpoints.
This is our index fields architecture.
We have used Azure OpenAI over the AI search indexer and deployed the bot as a web app using Azure Chat playground built in feature.
Our OpenAI parameters:
Strictness: 3
Retrieved documents: 8
temperature: 0
Top P :1
With the above application we are facing few issues such as:
- An Error occurred. Answers can't be saved this time. If the problem persists please contact administrator. or There was an error generating a response....
We have looked into multiple QnA threads for the above issue in both Microsoft and OpenAI forums but couldn't get a solution which is completely solving the issue.
- For the same question we are getting proper answer at sometime and we are shown with response: The requested information is not found in the retrieved data. Please try another query or topic.
We were expecting a slight difference in response format but not any information about the question which is existing in the data and answered by the bot occasionally.
Please let us know if we can do anything to avoid above two issues.
Thank you!
Azure AI Search
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
Azure AI services
Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
{count} votes
-
romungi-MSFT 49,096 Reputation points Microsoft Employee Moderator2024-03-01T10:39:01.2133333+00:00 @08382095 With respect to Azure OpenAI parameters, you can decrease the strictness and increase the retrieved documents to get a response before it fails. I would suggest adding diagnostic logging setting on your resource before retrying the scenario to get more logs.
If possible, it might be easier to debug the error if you can raise a support case to work with service team on exact request id responses.
-
08382095 65 Reputation points
2024-03-04T13:32:37.7566667+00:00 Thank you for your suggestion, we have tried setting strictness to 1 and retrieved documents to 20. Where in we are getting information to the queries out of the data source even after setting the system message explicitly stating it to not answer anything out of data source, which is not preferred in our use case.
And increasing the retrieved documents to 20 increased the retrieval time but didn't help us much in the accuracy.
Please let us know if we can do anything more to improve.
Thank you
Sign in to comment
Sign in to answer