'Add your data' feature in azure open ai is working differently in Azure open ai studio and azure open ai api call

Question

Hi Team,

We are implementing azure open ai API in chatbot to use private document data.

We have uploaded the document through azure open ai studio and used the index name for accessing the data in API call.

If we are asking a question out of scope of the document then, azure open ai studio, it will respond , there is no answer in the retrieved document. But accessing the same index using API call is giving answers for the question that is not in the limit of document.

Below is the screenshot,

For this I have not configured any specific system prompts also.

I have tried the same below code in bot framework in c#, I am not getting the same behavior,

using Azure.AI.OpenAI;
2using Azure;
3
4// Azure OpenAI setup
5var apiBase = "openai.openai.azure.com/"; // Add your endpoint here
6var apiKey = Environment.GetEnvironmentVariable("OPENAI_API_KEY"); // Add your OpenAI API key here
7var deploymentId = "Chatbot-OpenAI"; // Add your deployment ID here
8
9// Azure AI Search setup
10var searchEndpoint = "s0-basic.search.windows.net"; // Add your Azure AI Search endpoint here
11var searchKey = Environment.GetEnvironmentVariable("SEARCH_KEY"); // Add your Azure AI Search admin key here
12var searchIndexName = "openai"; // Add your Azure AI Search index name here
13var client = new OpenAIClient(new Uri(apiBase), new AzureKeyCredential(apiKey!));
14
15var chatCompletionsOptions = new ChatCompletionsOptions()
16{
17    Messages =
18    {
19        new ChatMessage(ChatRole.System, "What are the differences between Azure Machine Learning and Azure AI services?")
20    },
21    // The addition of AzureChatExtensionsOptions enables the use of Azure OpenAI capabilities that add to
22    // the behavior of Chat Completions, here the "using your own data" feature to supplement the context
23    // with information from an Azure AI Search resource with documents that have been indexed.
24    AzureExtensionsOptions = new AzureChatExtensionsOptions()
25    {
26        Extensions =
27        {
28            new AzureCognitiveSearchChatExtensionConfiguration()
29            {
30                SearchEndpoint = new Uri(searchEndpoint),
31                IndexName = searchIndexName,
32                SearchKey = new AzureKeyCredential(searchKey!),
33                QueryType = new AzureChatExtensionType(AzureCognitiveSearch),
34                Parameters = FromString([object Object]),
35            },
36        },
37    },
38    DeploymentName = Chatbot-OpenAI,
39    MaxTokens = 800,
40    StopSequences = null,
41    Temperature = 0,
42    
43
44};
45
46var response = await client.GetChatCompletionsAsync(
47    deploymentId,
48    chatCompletionsOptions);

Only change in our code is e are not using query type parameter

Can anyone help to understand how can i make api call better to get only document data answer response from the above API

Answer 1

@lakshmi Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

Within your Azure Open AI Studio, you seem to have enabled the below setting, to limit response to your own data:

From your application's API call, If you want to ensure that the responses from the API are limited to the content of the grounding data only, then you need to rely on the ShouldRestrictResultScope parameter. You need to set this to true within the AzureSearchChatExtensionConfiguration class.

More info ***here *and here.

Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help.