How can I prevent the Azure OpenAI chat from adding unnecessary details to its responses when asking questions on my documents?

Question

How can I prevent the Azure OpenAI chat from adding unnecessary details to its responses when asking questions on my documents?

AMROUN Lysa 396

I'm using the azure openai chat playground on my documents and I'm doing test questions, I notice that when I ask a question on a document the chat answers me correctly but adds other details of the document that are not necessary to the question asked, how can I fix this?

thks

lysa

VenkateshDodda-MSFT 25,241 Reputation points Microsoft Employee Moderator

2024-05-24T11:05:02.2866667+00:00
@AMROUN Lysa Thank you for posting your question in Microsoft Q&A, apologize for any inconvenience caused on this.

Could you please help more details about this issue to understand better.

What is the expected output and current results? help us with an sample and screenshots

Also help us is there any documentation if you are following.

What is type of index you are using in search service and searching technique as well to understand better.
VenkateshDodda-MSFT 25,241 Reputation points Microsoft Employee Moderator

2024-05-28T01:32:31.77+00:00

Hello @AMROUN Lysa ,

Following up to see if you have a chance to check the previous responses and helped. Do let us know if you have any further questions on this.
AMROUN Lysa 396 Reputation points

2024-05-28T06:02:04.1233333+00:00

VenkateshDodda-MSFT

thanks for the answer,

Yes, I had activated it. Could you tell me how to set these two values, because I have left them at 3 and 5 by default but I don't know what will work best to get the maximum number of correct answers.

thank you

lysa
AshokPeddakotla-MSFT 36,006 Reputation points Moderator

2024-05-29T03:24:25.36+00:00

AMROUN Lysa Please check my below response and let me know if you have any further queries.

Answer accepted by question author

0 additional answers

Your answer

VenkateshDodda-MSFT 25,241 Reputation points Microsoft Employee Moderator

2024-05-24T11:05:02.2866667+00:00

@AMROUN Lysa Thank you for posting your question in Microsoft Q&A, apologize for any inconvenience caused on this.

Could you please help more details about this issue to understand better.

What is the expected output and current results? help us with an sample and screenshots

Also help us is there any documentation if you are following.

What is type of index you are using in search service and searching technique as well to understand better.
VenkateshDodda-MSFT 25,241 Reputation points Microsoft Employee Moderator

2024-05-28T01:32:31.77+00:00

Hello @AMROUN Lysa ,

Following up to see if you have a chance to check the previous responses and helped. Do let us know if you have any further questions on this.
AMROUN Lysa 396 Reputation points

2024-05-28T06:02:04.1233333+00:00

VenkateshDodda-MSFT

thanks for the answer,

Yes, I had activated it. Could you tell me how to set these two values, because I have left them at 3 and 5 by default but I don't know what will work best to get the maximum number of correct answers.

thank you

lysa
AshokPeddakotla-MSFT 36,006 Reputation points Moderator

2024-05-29T03:24:25.36+00:00

AMROUN Lysa Please check my below response and let me know if you have any further queries.

Answer 1

AshokPeddakotla-MSFT 36,006 Moderator

AMROUN Lysa Greetings!

I'm using the azure openai chat playground on my documents and I'm doing test questions, I notice that when I ask a question on a document the chat answers me correctly but adds other details of the document that are not necessary to the question asked, how can I fix this?

Did you enable Limit responses to your data setting?

When Limit responses to your data setting is disabled, the model supplements its responses with its own knowledge in addition to your documents. When this setting is enabled, the model attempts to only rely on your documents for responses.

Screenshot of the advanced settings.

Also, you need to train the model and use correct system message. See Prompt engineering techniques for more details.

Do let me know if you have any further queries.

AshokPeddakotla-MSFT 36,006 Reputation points Moderator

2024-05-29T03:23:47.17+00:00

AMROUN Lysa If you are referring to the Strictness and Retrieved documents settings, please see below for more details.

Strictness : Determines the system's aggressiveness in filtering search documents based on their similarity scores. The system queries Azure Search or other document stores, then decides which documents to provide to large language models like ChatGPT. Filtering out irrelevant documents can significantly enhance the performance of the end-to-end chatbot. Some documents are excluded from the top-K results if they have low similarity scores before forwarding them to the model. This is controlled by an integer value ranging from 1 to 5. Setting this value to 1 means that the system will minimally filter documents based on search similarity to the user query. Conversely, a setting of 5 indicates that the system will aggressively filter out documents, applying a very high similarity threshold. If you find that the chatbot omits relevant information, lower the filter's strictness (set the value closer to 1) to include more documents. Conversely, if irrelevant documents distract the responses, increase the threshold (set the value closer to 5). This is the strictness parameter in the API, and set to 3 by default.

Retrieved documents : This parameter is an integer that can be set to 3, 5, 10, or 20, and controls the number of document chunks provided to the large language model for formulating the final response. By default, this is set to 5. The search process can be noisy and sometimes, due to chunking, relevant information might be spread across multiple chunks in the search index. Selecting a top-K number, like 5, ensures that the model can extract relevant information, despite the inherent limitations of search and chunking. However, increasing the number too high can potentially distract the model. Additionally, the maximum number of documents that can be effectively used depends on the version of the model, as each has a different context size and capacity for handling documents. If you find that responses are missing important context, try increasing this parameter. This is the topNDocuments parameter in the API, and is 5 by default.

Please check Azure OpenAI On Your Data and Learn how to generate or manipulate text for more details.

Also, see how to Improve Performance and latency of the models.

Do let me know if that helps or have any other queries.

If the response helped, please do click Accept Answer and Yes for was this answer helpful.

Doing so would help other community members with similar issue identify the solution. I highly appreciate your contribution to the community.
AMROUN Lysa 396 Reputation points

2024-05-29T06:30:42.8466667+00:00

@AshokPeddakotla-MSFT

Thank you for your answer , it's very helping :)
AshokPeddakotla-MSFT 36,006 Reputation points Moderator

2024-05-29T10:32:54.3666667+00:00

AMROUN Lysa You're welcome!

Glad to hear it was helpful.

Share via

How can I prevent the Azure OpenAI chat from adding unnecessary details to its responses when asking questions on my documents?

0 additional answers

Your answer