Azure OpenAI with own data (AI Search)

Question

I've explored Azure OpenAI and aim to integrate the "Add your data" feature with AI Search, which is no longer in preview. I'm attempting to test if I can directly call the API and incorporate it into my solution. Despite searching online for sample code, I'm uncertain about how to add configurations such as stop sequences, temperature, and frequency penalty. I understand that I need to connect Azure Blob Storage with my AI Search and integrate it into my OpenAI model. The existing code functions well; however, I'm curious about any potential omissions or additional configurations I could apply. Currently, I'm experimenting with C#, although I will switch to VB.NET for integration into my solution. I'm using Azure.OpenAI version 2.0.0-beta.2. My goal is to create a chatbot that will handle responses and queries within my solution, akin to a chat system.

I have another question, If I wanted to integrate into my solution, would I make it as an API to call, or straight implement into my solution backend? If possible, it is best to compatible for both mobile and web-based solution.

PS: My ChatCompletionOptions only have these properties: AllowPartialResults, FieldMappings, Inscope, MaxSearchQueries, OutputContextFlags, QueryType, RoleInformation, SemanticConfiguration, VectorizationSource.

#pragma warning disable AOAI001
using System.Text;
using Azure;
using Azure.AI.OpenAI;
using Azure.AI.OpenAI.Chat;
using OpenAI.Chat;
using static System.Environment;
string azureOpenAIEndpoint = GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT");
string azureOpenAIKey = GetEnvironmentVariable("AZURE_OPENAI_API_KEY");
string deploymentName = GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_ID");
string searchEndpoint = GetEnvironmentVariable("AZURE_AI_SEARCH_ENDPOINT");
string searchKey = GetEnvironmentVariable("AZURE_AI_SEARCH_API_KEY");
string searchIndex = GetEnvironmentVariable("AZURE_AI_SEARCH_INDEX");
AzureOpenAIClient azureClient = new(
    new Uri(azureOpenAIEndpoint),
    new AzureKeyCredential(azureOpenAIKey));
ChatClient chatClient = azureClient.GetChatClient(deploymentName);
ChatCompletionOptions options = new();
options.AddDataSource(new AzureSearchChatDataSource()
{
    Endpoint = new Uri(searchEndpoint),
    IndexName = searchIndex,
    Authentication = DataSourceAuthentication.FromApiKey(searchKey),
    Strictness = 4,
    TopNDocuments = 4,
});
while (true)
{
    Console.Write("Enter your question (or type 'exit' to quit): ");
    string userInput = Console.ReadLine();
    if (userInput.Equals("exit", StringComparison.OrdinalIgnoreCase))
    {
        break;
    }
    var chatUpdates = chatClient.CompleteChatStreamingAsync(
        new[]
        {
            new UserChatMessage(userInput),
        }, options);
    AzureChatMessageContext onYourDataContext = null;
    await foreach (var chatUpdate in chatUpdates)
    {
        if (chatUpdate.Role.HasValue)
        {
            Console.WriteLine($"{chatUpdate.Role}: ");
        }
        var chatResponseBuilder = new StringBuilder();
        foreach (var contentPart in chatUpdate.ContentUpdate)
        {
            Console.Write(contentPart.Text);
            await Task.Delay(TimeSpan.FromMilliseconds(50));
        }
        if (onYourDataContext == null)
        {
            onYourDataContext = chatUpdate.GetAzureMessageContext();
        }
    }
    Console.WriteLine();
    if (onYourDataContext?.Intent != null)
    {
        Console.WriteLine($"Intent: {onYourDataContext.Intent}");
    }
    foreach (AzureChatCitation citation in onYourDataContext?.Citations ?? Array.Empty<AzureChatCitation>())
    {
        Console.WriteLine($"Citation: {citation.Content}");
    }
}

Answer 1

@Tom Chow Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

Question 1:
To add configurations such as stop sequences, temperature, and frequency penalty.

Answer:
You can modify your ChatCompletionOptions object. Here’s an example of how you can include these parameters:

ChatCompletionOptions options = new()

{

    Temperature = 0.7,

    MaxTokens = 150,

    FrequencyPenalty = 0.5,

    PresencePenalty = 0.0,

    StopSequences = new List<string> { "\n", "User:" }

};

Regarding connecting Azure Blob Storage with your AI Search and integrating it into your OpenAI model, you need to ensure that your data is properly indexed and accessible. You can follow the steps to connect your data source using Azure OpenAI Studio. Make sure to enable Cross-origin resource sharing (CORS) for your Azure Blob Storage. More info here and here.

. .

. Question 2: Integrating your chatbot into your solution.

.

Answer: You have two main options shared below, Both approaches can be compatible with mobile and web-based solutions. Using an API might offer more flexibility and scalability

API Call: Create an API that your solution can call. This is a flexible approach and can be used for both mobile and web-based solutions.

Direct Implementation: Integrate the chatbot directly into your backend. This might be simpler for some use cases but could be less flexible than an API.

. . .

Question 3: Customize the responses for questions that are not related to the provided documents .

.

Answer: This can be done by setting specific instructions in the RoleInformation property of your ChatCompletionOptions. For example:

options.RoleInformation = "As a support representative of 'this company', I am here to provide any information related to it. If you require further assistance, would you like to be connected to an agent?";

This customization can be done using the GPT-4o model or other models available in Azure OpenAI Service. You don’t necessarily need custom models for this; you can achieve it through prompt engineering and setting appropriate instructions. . .

Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help. ** Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.