Inconsistent Citations using Azure Open AI on my own Data

Santos, Miguel Ângelo 0 Reputation points
2024-05-23T09:42:15.97+00:00

Good morning

I'm creating a chatbot on my own data using gpt-35-turbo and Azure AI Search with query_type = Hybrid + Semantic. The index was created on Azure OAI Studio and my files are all pdfs, stored in a Azure Blob Storage. Currently, I'm using my chatbot on the webapp created by the deploy on the Azure OAI Studio.

If relevant

  • my documents are in Portuguese and my queries/questions are also in Portuguese
  • OAI : temperature = 0 and top_P=0
  • AI Search : top_K = 5 and Strictness = 3

The problem here is that, for the same query, I get different citations from AI Search. For example, for the exactly same question

  1. the chatbot some times returns 1 citation
  2. the chatbot some times returns 2 citations (where 1 citation is common to both tries)
  3. the chatbot some times returns 1 citation completely different than the others

Any idea on how I can make it more deterministic ? In other words, how can I replicate results?

Azure AI Search
Azure AI Search
An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.
810 questions
Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,494 questions
{count} votes