I'm using the azure openai chat playground on my documents and I'm doing test questions, I notice that when I ask a question on a document the chat answers me correctly but adds other details of the document that are not necessary to the question asked, how can I fix this? thks lysa

AMROUN Lysa Greetings! I'm using the azure openai chat playground on my documents and I'm doing test questions, I notice that when I ask a question on a document the chat answers me correctly but adds other details of the document that are not necessary to the question asked, how can I fix this? Did you enable Limit responses to your data setting? When Limit responses to your data setting is disabled, the model supplements its responses with its own knowledge in addition to your documents. When this setting is enabled, the model attempts to only rely on your documents for responses. Also, you need to train the model and use correct system message. See Prompt engineering techniques for more details. Do let me know if you have any further queries.

How can I prevent the Azure OpenAI chat from adding unnecessary details to its responses when asking questions on my documents?

Accepted answer

AshokPeddakotla-MSFT 34,616 Reputation points

2024-05-27T03:08:19.1766667+00:00

AMROUN Lysa Greetings!

I'm using the azure openai chat playground on my documents and I'm doing test questions, I notice that when I ask a question on a document the chat answers me correctly but adds other details of the document that are not necessary to the question asked, how can I fix this?

Did you enable Limit responses to your data setting?

When Limit responses to your data setting is disabled, the model supplements its responses with its own knowledge in addition to your documents. When this setting is enabled, the model attempts to only rely on your documents for responses.

Also, you need to train the model and use correct system message. See Prompt engineering techniques for more details.

Do let me know if you have any further queries.
Please sign in to rate this answer.

1 person found this answer helpful.
AshokPeddakotla-MSFT 34,616 Reputation points

2024-05-29T03:23:47.17+00:00

AMROUN Lysa If you are referring to the Strictness and Retrieved documents settings, please see below for more details.

Strictness : Determines the system's aggressiveness in filtering search documents based on their similarity scores. The system queries Azure Search or other document stores, then decides which documents to provide to large language models like ChatGPT. Filtering out irrelevant documents can significantly enhance the performance of the end-to-end chatbot. Some documents are excluded from the top-K results if they have low similarity scores before forwarding them to the model. This is controlled by an integer value ranging from 1 to 5. Setting this value to 1 means that the system will minimally filter documents based on search similarity to the user query. Conversely, a setting of 5 indicates that the system will aggressively filter out documents, applying a very high similarity threshold. If you find that the chatbot omits relevant information, lower the filter's strictness (set the value closer to 1) to include more documents. Conversely, if irrelevant documents distract the responses, increase the threshold (set the value closer to 5). This is the strictness parameter in the API, and set to 3 by default.

Retrieved documents : This parameter is an integer that can be set to 3, 5, 10, or 20, and controls the number of document chunks provided to the large language model for formulating the final response. By default, this is set to 5. The search process can be noisy and sometimes, due to chunking, relevant information might be spread across multiple chunks in the search index. Selecting a top-K number, like 5, ensures that the model can extract relevant information, despite the inherent limitations of search and chunking. However, increasing the number too high can potentially distract the model. Additionally, the maximum number of documents that can be effectively used depends on the version of the model, as each has a different context size and capacity for handling documents. If you find that responses are missing important context, try increasing this parameter. This is the topNDocuments parameter in the API, and is 5 by default.

Please check Azure OpenAI On Your Data and Learn how to generate or manipulate text for more details.

Also, see how to Improve Performance and latency of the models.

Do let me know if that helps or have any other queries.

If the response helped, please do click Accept Answer and Yes for was this answer helpful.

Doing so would help other community members with similar issue identify the solution. I highly appreciate your contribution to the community.

AMROUN Lysa 361 Reputation points

2024-05-29T06:30:42.8466667+00:00

@AshokPeddakotla-MSFT

Thank you for your answer , it's very helping :)

AshokPeddakotla-MSFT 34,616 Reputation points

2024-05-29T10:32:54.3666667+00:00

AMROUN Lysa You're welcome!

Glad to hear it was helpful.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

How can I prevent the Azure OpenAI chat from adding unnecessary details to its responses when asking questions on my documents?

0 additional answers

Your answer