Implementing Azure OpenAI in C# .NET for Custom Data Integration from OneDrive, SharePoint, Azure Cognitive Search, and Blob Storage.

Question

Implementing Azure OpenAI in C# .NET for Custom Data Integration from OneDrive, SharePoint, Azure Cognitive Search, and Blob Storage.

REDDY, REDDYVALLA NAVEENKUMAR 115

Hi Team,

I am interested in utilizing Azure OpenAI with custom data from various sources such as OneDrive/SharePoint files, local files, Azure Cognitive Search, and Blob Storage.

While searching for information, I came across a sample on https://github.com/Azure-Samples/azure-search-openai-demo/, but it is implemented in Python. I would like to understand how this works in C# .NET. Thank you.

romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2023-06-23T12:12:19.79+00:00

@REDDY, REDDYVALLA NAVEENKUMAR The repo mentioned uses a storage account and a search resource to deploy a webapp that can be used with Azure OpenAI. The implementation of this repo is now actually available directly under Azure OpenAI studio as a preview feature with a no code experience by configuring the storage and search resources and deploying a webapp.

If you would like to still build an app with C# a similar working solution is currently not published but you can use Azure SDK for .NET for storage and search to deploy and configure the services before using Azure OpenAI REST API endpoint.
REDDY, REDDYVALLA NAVEENKUMAR 115 Reputation points

2023-06-23T12:38:43.7366667+00:00

Thanks for the Information. @romungi-MSFT

We are creating an Azure chatbot that functions as an FAQ bot. The goal is to provide users with relevant information based on their queries. The chatbot will gather data from various sources, including OneDrive/SharePoint files, local files, Azure Cognitive Search, and Blob Storage. By using Azure Open AI, the chatbot will process this data and generate accurate responses to user queries.

Do we have any sample for the same. I would like to understand how this works in C# .NET. Thank you.
Siry, Gaetan 220 Reputation points

2023-06-24T03:02:03.8833333+00:00

@REDDY, REDDYVALLA NAVEENKUMAR I am in the same boat - I just added data through Azure AI Studio, and he Azure AI Studio chat playground can respond to questions about the data - but my current model has no idea about the data that I just loaded and the sample deployable app is not a .net app which does not help, since the solution that we build and that talks to OpenAI is a .NET app. Disappointing to say the least.
Lewis Thrasher 20 Reputation points

2023-06-26T15:50:33.66+00:00

@romungi-MSFT I am also looking for an answer to this. To re-phrase the issue from what I can tell. The C# Azure AI SDK hasn't been updated to include the datasource parameters that are needed to tell the service to use the new "Use your own data" info. It looks like the only option right now if you use the SDK is to convert the calls back into Rest API formats and manually handle the responses. Do we have any idea on when the SDK will be updated?
Travis Wilson 115 Reputation points Microsoft Employee

2023-06-26T18:36:35.8633333+00:00

Hello! You're completely correct that the Azure OpenAI client libraries don't yet support the "Use your own data" scenario (and, more generally, the /extensions/chat/completions REST endpoint). That integration is an area we're actively working on and, while we don't yet have an ETA to report, we're working to bring it to the libraries as soon as we can. There's a bit of extra complexity introduced by schema divergence and a desire for us to keep things tidy in light of OpenAI's Functions support going live, so we appreciate the patience while we work on getting it right.
Julio Saenz 0 Reputation points Microsoft Employee

2023-06-29T19:20:27.7633333+00:00

I had been scratching my head around this, i cold not fins a way to easily integrate the search results from cognitive search into call to the model. Question here, can i pass in the docs as "Few shot" examples? or how can we get around this issue?

3 answers

Your answer

romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2023-06-23T12:12:19.79+00:00

@REDDY, REDDYVALLA NAVEENKUMAR The repo mentioned uses a storage account and a search resource to deploy a webapp that can be used with Azure OpenAI. The implementation of this repo is now actually available directly under Azure OpenAI studio as a preview feature with a no code experience by configuring the storage and search resources and deploying a webapp.

If you would like to still build an app with C# a similar working solution is currently not published but you can use Azure SDK for .NET for storage and search to deploy and configure the services before using Azure OpenAI REST API endpoint.
REDDY, REDDYVALLA NAVEENKUMAR 115 Reputation points

2023-06-23T12:38:43.7366667+00:00

Thanks for the Information. @romungi-MSFT

We are creating an Azure chatbot that functions as an FAQ bot. The goal is to provide users with relevant information based on their queries. The chatbot will gather data from various sources, including OneDrive/SharePoint files, local files, Azure Cognitive Search, and Blob Storage. By using Azure Open AI, the chatbot will process this data and generate accurate responses to user queries.

Do we have any sample for the same. I would like to understand how this works in C# .NET. Thank you.
Siry, Gaetan 220 Reputation points

2023-06-24T03:02:03.8833333+00:00

@REDDY, REDDYVALLA NAVEENKUMAR I am in the same boat - I just added data through Azure AI Studio, and he Azure AI Studio chat playground can respond to questions about the data - but my current model has no idea about the data that I just loaded and the sample deployable app is not a .net app which does not help, since the solution that we build and that talks to OpenAI is a .NET app. Disappointing to say the least.
Lewis Thrasher 20 Reputation points

2023-06-26T15:50:33.66+00:00

@romungi-MSFT I am also looking for an answer to this. To re-phrase the issue from what I can tell. The C# Azure AI SDK hasn't been updated to include the datasource parameters that are needed to tell the service to use the new "Use your own data" info. It looks like the only option right now if you use the SDK is to convert the calls back into Rest API formats and manually handle the responses. Do we have any idea on when the SDK will be updated?
Travis Wilson 115 Reputation points Microsoft Employee

2023-06-26T18:36:35.8633333+00:00

Hello! You're completely correct that the Azure OpenAI client libraries don't yet support the "Use your own data" scenario (and, more generally, the /extensions/chat/completions REST endpoint). That integration is an area we're actively working on and, while we don't yet have an ETA to report, we're working to bring it to the libraries as soon as we can. There's a bit of extra complexity introduced by schema divergence and a desire for us to keep things tidy in light of OpenAI's Functions support going live, so we appreciate the patience while we work on getting it right.
Julio Saenz 0 Reputation points Microsoft Employee

2023-06-29T19:20:27.7633333+00:00

I had been scratching my head around this, i cold not fins a way to easily integrate the search results from cognitive search into call to the model. Question here, can i pass in the docs as "Few shot" examples? or how can we get around this issue?

Answer 1

Lewis Thrasher 20

So for anyone still looking for this. Going through the playground and hitting "View Code" does not show the correct code that utilizes the new custom datasource. However, I was able to find the documentation on what you need to send in the REST command to get it. If you go here, https://learn.microsoft.com/en-us/azure/cognitive-services/openai/use-your-data-quickstart?source=recommendations&tabs=powershell&pivots=rest-api . This will show you the endpoint, and explain the variables needed, but the body is also incomplete in this example. It doesn't show you how to send the field mapping information. To get that I found it here. https://github.com/Azure/azure-rest-api-specs/blob/main/specification/cognitiveservices/data-plane/AzureOpenAI/inference/preview/2023-06-01-preview/inference.json . If you scroll down to line 1139 you can see an example of the full body that you need to send to the new /extensions/chat/completions endpoint. Here is the example

{		
		"dataSources": [
            {
              "type": "AzureCognitiveSearch",
              "parameters": {
                "endpoint": "https://mysearchexample.search.windows.net",
                "key": "***(admin key)",
                "indexName": "my-chunk-index",
                "fieldsMapping": {
                  "titleField": "productName",
                  "urlField": "productUrl",
                  "filepathField": "productFilePath",
                  "contentFields": [
                    "productDescription"
                  ],
                  "contentFieldsSeparator": "\n"
                },
                "topNDocuments": 5,
                "queryType": "semantic",
                "semanticConfiguration": "defaultConfiguration",
                "inScope": true,
                "roleInformation": "roleInformation"
              }
            }
          ],
          "messages": [
            {
              "role": "user",
              "content": "Where can I find a hiking place in Seattle?"
            }
          ],
          "temperature": 0.9
}

I have tested this in my c# app and using postman and it is working.

Ziggy Zulueta 495 Reputation points MVP

2023-07-08T16:38:52.09+00:00

Your answer is the same as mine in #2. In fact, you can create your own website as well by calling REST. All you need to do is to keep on adding the User and Assistant roles in the messages array to keep the history of the chat and for the model to use the history when making a response.
Lewis Thrasher 20 Reputation points

2023-07-08T16:46:15.8433333+00:00

It actually isn't the same. If you look at the sample code from your answer, it is not using the new extension REST endpoint to use the datasource info. So it will be just a normal chat bot that does not use your own data. That is the problem with using the "View Code" option in the studio. While you can add the custom datasource in the studio and it works there, it does not generate the correct code.

Brett 0

Nice work. Looks good but I get this message from Postman:


    "error": {
        "message": "Unrecognized request argument supplied: dataSources",
        "type": "invalid_request_error",
        "param": null,
        "code": null
    }
}

Here is the body of my request.

{"dataSources": [
            {
              "type": "AzureCognitiveSearch",
              "parameters": {
                "endpoint": "https://xxxxxxx-search.search.windows.net",
                "key": "xxxxx",
                "indexName": "xxxxx-index",
                "contentFieldsSeparator": "\n",
                "topNDocuments": 5,
                "roleInformation": "roleInformation"
              }
            }
          ],
  "messages": [
    {
      "role": "system",
      "content": "Your name is Bill."
    },
    {
      "role": "user",
      "content": "What is your name?"
    }
  ],
  "temperature": 0,
  "top_p": 1,
  "frequency_penalty": 0,
  "presence_penalty": 0,
  "max_tokens": 800,
  "stop": null
}

Would you mind sharing your C# code as well for how to implement this?

Lewis Thrasher 20 Reputation points

2023-07-12T13:44:52.4266667+00:00

I will look at adding some code, but right off the bat I can see some problems. With this endpoint you don't add a "System" message at the beginning. Instead you add that information in the "roleInformation" parameter. Also, from what I can tell you dont send the top_p,frequency_penalty,presence_penalty,max_tokens,stop parameters at all. Your body structure should look exactly like I posted. Also make sure you are following the instructions from this page and sending to the correct endpoint and with all the headers/url parameters. https://learn.microsoft.com/en-us/azure/cognitive-services/openai/use-your-data-quickstart?source=recommendations&tabs=powershell&pivots=rest-api. Remember, this is specifically to send it questions that it will answer based on the documents it can pull from the Cognitive Search Index. It wont answer general questions like it would using the normal endpoint.

Lewis Thrasher 20

Here is an example from Postman.

URL: https://$AZUREOPENAIURL/openai/deployments/$DEPLOYMENTMODEL/extensions/chat/completions?api-version=2023-06-01-preview

Headers:

api-key: $key

chatgpt_url: $AZUREOPENAIURL (same as above)

chatgpt_key: $key

Body:

{		
		"dataSources": [
            {
              "type": "AzureCognitiveSearch",
              "parameters": {
                "endpoint": "https://mysearchexample.search.windows.net",
                "key": "***(admin key)",
                "indexName": "my-chunk-index",
                "fieldsMapping": {
                  "titleField": "productName",
                  "urlField": "productUrl",
                  "contentFields": [
                    "productDescription"
                  ],
                  "contentFieldsSeparator": "\n"
                },
                "topNDocuments": 3,
                "roleInformation": "You are an AI Assistant named Bill"
              }
            }
          ],
          "messages": [
            {
              "role": "user",
              "content": "SOME QUESTION RELATED TO ITEMS FOUND IN SEARCH INDEX"
            }
          ],
          "temperature": 0.9
}

User's image

Lewis Thrasher 20

Here is a quick and dirty C# method with the classes to send the request and get a response back. I just threw the json objects for the request and response in a Json to C# converter. This method is assuming you are injecting a httpclientfactory, you can create the client manually too for testing if you need.

using System.Text.Json.Serialization;
    
	public partial class GPTBody
    {
        [JsonPropertyName("dataSources")]
        public List<DataSource> DataSources { get; set; }

        [JsonPropertyName("messages")]
        public List<Message> Messages { get; set; }

        public partial class DataSource
        {
            [JsonPropertyName("type")]
            public string DataType { get; set; }

            [JsonPropertyName("parameters")]
            public Parameters Parameters { get; set; }

            [JsonPropertyName("roleInformation")]
            public string RoleInformation { get; set; }

            [JsonPropertyName("topNDocuments")]
            public int TopNDocuments { get; set; }
        }

        public partial class FieldsMapping
        {
            [JsonPropertyName("titleField")]
            public string TitleField { get; set; }

            [JsonPropertyName("urlField")]
            public string UrlField { get; set; }

            [JsonPropertyName("filepathField")]
            public string FilepathField { get; set; }

            [JsonPropertyName("contentFields")]
            public List<string> ContentFields { get; set; }

            [JsonPropertyName("contentFieldsSeparator")]
            public string ContentFieldsSeparator { get; set; }
        }

        public partial class Parameters
        {
            [JsonPropertyName("endpoint")]
            public string Endpoint { get; set; }

            [JsonPropertyName("key")]
            public string Key { get; set; }

            [JsonPropertyName("indexName")]
            public string IndexName { get; set; }

            [JsonPropertyName("fieldsMapping")]
            public FieldsMapping FieldsMapping { get; set; }
        }

        public partial class Message
        {
            [JsonPropertyName("role")]
            public string Role { get; set; }

            [JsonPropertyName("content")]
            public string Content { get; set; }
        }
    }
	
	public partial class GPTResponse
    {
        [JsonPropertyName("id")]
        public string Id { get; set; }

        [JsonPropertyName("model")]
        public string Model { get; set; }

        [JsonPropertyName("created")]
        public long Created { get; set; }

        [JsonPropertyName("object")]
        public string Object { get; set; }

        [JsonPropertyName("choices")]
        public Choice[] Choices { get; set; }

        public partial class Choice
        {
            [JsonPropertyName("index")]
            public long Index { get; set; }

            [JsonPropertyName("messages")]
            public Message[] Messages { get; set; }
        }

        public partial class Message
        {
            [JsonPropertyName("role")]
            public string Role { get; set; }

            [JsonPropertyName("content")]
            public string Content { get; set; }
            [JsonPropertyName("end_turn")]
            public bool EndTurn { get; set; }
        }
    }




private async Task<GPTResponse> SendGPTMessage() {
  var options = new JsonSerializerOptions(JsonSerializerDefaults.Web);
  options.Converters.Add(new JsonStringEnumConverter());
  
  var httpClient = _httpClientFactory.CreateClient();
  var request = new HttpRequestMessage(HttpMethod.Post, $"https://{AzureOpenAIUrl}/openai/deployments/{gptModel}/extensions/chat/completions?api-version=2023-06-01-preview");
  httpClient.DefaultRequestHeaders.Add("api-key", api-key);
  httpClient.DefaultRequestHeaders.Add("chatgpt_url", AzureOpenAIUrl);
  httpClient.DefaultRequestHeaders.Add("chatgpt_key", api-key);
  
  var body = new GPTBody() {
    DataSources = new List<DataSource> () {
        new DataSource {
          DataType = "AzureCognitiveSearch",
          Parameters = new Parameters {
			Endpoint = SearchURL,
			Key = searchIndexKey,
			IndexName = "index",
			FieldsMapping = new FieldsMapping {
				TitleField = "title",
				UrlField = "url",
				ContentFields = new List < string > {
					"content"
				},
            ContentFieldsSeparator = "\n",
			}
			},
          RoleInformation = "You are an AI assistant.",
          TopNDocuments = 2
        }
        },
	Messages = new List<Message>{
		new Message{
			Role = "user",
			Content = "SOME QUESTION RELATED TO ITEMS FOUND IN SEARCH INDEX"
		}
	}
  };
  
  var json = System.Text.Json.JsonSerializer.Serialize(body, options);
  request.Content = new StringContent(json, Encoding.UTF8, "application/json");
  var response = await httpClient.SendAsync(request);
  if (response.IsSuccessStatusCode) {
    return await System.Text.Json.JsonSerializer.DeserializeAsync<GPTResponse>
      (await response.Content.ReadAsStreamAsync(), new JsonSerializerOptions() {PropertyNameCaseInsensitive = true});
  }
  
  throw new HttpRequestException(await response.Content.ReadAsStringAsync());
}

Jacob Thomas 55 Reputation points

2024-07-26T15:54:01.18+00:00

Has anyone gotten this working with the Azure OpenAI .NET library? The fields don't seem to match the json structure on a one to one level. The library documentation hardly exists too

Answer 2

Ziggy Zulueta 495 MVP

@REDDYVALLA NAVEENKUMAR REDDY

Let me try to break down your question a bit: I am interested in utilizing Azure OpenAI with custom data from various sources such as OneDrive/SharePoint files, local files, Azure Cognitive Search, and Blob Storage.

From what we see, there is only one data source right now that you can link your Azure Open AI

Give this, You would have to put all your data in Azure Cognitive Search.

Second you needed a C# implementation yes? I see two ways to do this:

1.There is a sample code available now in the studio:

You can create your C# application then call the Azure OpenAI via API calls. I tested this myself via Postman. You can see a sample of the call by checking the curl

I hope this help you in your journey. Please accept this answer if this helps you.

Syedbabar Ali 400 Reputation points

2023-07-06T06:05:42.1466667+00:00

I'm also interested in creating a chatbot using my own data with Azure Open AI.

However, when I obtained the code from the Chat Playground's "View Code" button, it didn't seem to read the data I uploaded in the Chat Playground. Additionally, the c# code did not mention any Blob Storage or Azure Cognitive Storage names.

Which work like normal chatgpt 3.

As a C# .NET developer, I'm looking for sample code in either C# or Python that utilizes Azure Open AI and accepts input based on my organization's policies.

If you have any code snippets or examples, please help.

Thank you for your response. I appreciate your guidance, as it has enabled me to successfully utilize Azure Open AI in C# using Visual Studio.

Answer 3

Gabriele Simula 50

for me was the URL wrong.

I was using

.../openai/deployments/gpt-turbo/chat/completions?api-version=2023-07-01-preview

instead this is the right url

.../openai/deployments/gpt-turbo/extensions/chat/completions?api-version=2023-07-01-preview

Boris Brodsky 0 Reputation points

2023-07-16T19:34:26.9833333+00:00

For those interested in making API calls against ChatGPT grounded with your own data, and while MS is developing proper SDK integration, you can use my Azure Function, which wraps REST API in an effortless way: https://borisbrodsky.com/azure/api-wrapper-to-integrate-azure-openai-chatgpt-model-with-your-own-data-over-api/

Share via

Implementing Azure OpenAI in C# .NET for Custom Data Integration from OneDrive, SharePoint, Azure Cognitive Search, and Blob Storage.

3 answers

Your answer