Improving accuracy of queries in Azure OpenAI and Cognitive Search

Leonhard Blank 5 Reputation points

I have a blob storage with around 1500 accounting documents indexed using Cognitive Search, with OCR skills integration. Although the index seems fine, the model's query results seem inaccurate. For example, when performing the query "Tell me everything about the document where the date is 20.09.2023" in the playground, the model responds "The requested information is not found in the retrieved data." To understand why this happens mostly, I set up a log analytics workspace and found the following log when filtering for the query executed by the model: " about document with date 20.09.2023...". The query is just a snippet of the question, without understanding the question itself. As a result, this query doesn't lead to any useful information. This problem occurs with almost every question, and I was wondering whether there is a way to make the chat perform better queries?

The model used is GPT 3.5 Turbo 0301. Both semantic and keyword search did have a problem with formulating accurate prompts,

Azure Cognitive Search
Azure Cognitive Search
An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.
497 questions
Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
1,395 questions
{count} vote