How to refresh url programatically in language studio using c#
I'm trying to refresh the url (I'm using excel sheet blob url) in c# to update the kb's.
Code :
using System.Net.Http; using System.Net.Http.Headers; HttpClient client = new HttpClient(); HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Patch, "https://{ENDPOINT}.api.cognitive.microsoft.com/language/query-knowledgebases/projects/{PROJECT-NAME}/sources?api-version=2021-10-01"); request.Headers.Add("Ocp-Apim-Subscription-Key", "{API-KEY}"); request.Content = new StringContent("[\n {\n "op": "replace",\n "value": {\n "displayName": "source5",\n "sourceKind": "url",\n "sourceUri": https://download.microsoft.com/download/7/B/1/7B10C82E-F520-4080-8516-5CF0D803EEE0/surface-book-user-guide-EN.pdf,\n "refresh": "true"\n }\n }\n]"); request.Content.Headers.ContentType = new MediaTypeHeaderValue("application/json"); HttpResponseMessage response = await client.SendAsync(request); response.EnsureSuccessStatusCode(); string responseBody = await response.Content.ReadAsString
I'm using this code in c#. I'm getting bad request as a response.
Azure AI services
-
VasaviLankipalle-MSFT 17,476 Reputation points
2023-03-21T20:03:57.0333333+00:00 Hi @KuppiReddygariSamthrushaReddy-3789 , Thanks for using Microsoft Q&A Platform.
Have you tried following this Refresh a URL programmatically documentation? https://learn.microsoft.com/en-us/azure/cognitive-services/language-service/question-answering/how-to/export-import-refresh#refresh-a-url-programmatically
Is this something you are looking for?
curl -X PATCH -H "Ocp-Apim-Subscription-Key: {API-KEY}" -H "Content-Type: application/json" -d '[ { "op": "replace", "value": { "displayName": "source5", "sourceKind": "url", "sourceUri": https://download.microsoft.com/download/7/B/1/7B10C82E-F520-4080-8516-5CF0D803EEE0/surface-book-user-guide-EN.pdf, "refresh": "true" } } ]' -i 'https://{ENDPOINT}.api.cognitive.microsoft.com/language/query-knowledgebases/projects/{PROJECT-NAME}/sources?api-version=2021-10-01'
Regards,
Vasavi -
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-03-23T08:16:04.4733333+00:00 yes I followed the same in C# instead of bash.
-
VasaviLankipalle-MSFT 17,476 Reputation points
2023-03-24T07:26:02.6033333+00:00 @KuppiReddygariSamthrushaReddy-3789 , can you please share the complete code script and the error response here?
-
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-03-27T07:52:45.55+00:00 this is the error response :
Code :
HttpClient client = new HttpClient(); HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Patch, "https://{ENDPOINT}.cognitiveservices.azure.com/language/query-knowledgebases/projects/{PROJECT-NAME}/sources?api-version=2021-10-01"); request.Headers.Add("Ocp-Apim-Subscription-Key", {API-KEY}); request.Content = new StringContent("[\n {\n \"op\": \"replace\",\n \"value\": {\n \"displayName\": \"source5\",\n \"sourceKind\": \"url\",\n \"sourceUri\": {URL},\n \"refresh\": \"true\"\n }\n }\n]"); request.Content.Headers.ContentType = new MediaTypeHeaderValue("application/json"); HttpResponseMessage response = await client.SendAsync(request); response.EnsureSuccessStatusCode(); string responseBody = await response.Content.ReadAsStringAsync(); return responseBody;
-
VasaviLankipalle-MSFT 17,476 Reputation points
2023-03-28T21:10:57.8666667+00:00 @KuppiReddygariSamthrushaReddy-3789 , Thanks for sharing this. Have you tried entering debug (F12 option) mode to check for errors?
-
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-03-29T06:30:09.7333333+00:00 @VasaviLankipalle-MSFT that is the only error I'm getting which I added above
-
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-03-30T06:35:16.15+00:00 @VasaviLankipalle-MSFT It's throwing badrequest in postman as well.
this is the request
{ "op": "replace", "value": { "displayName": "source5", "sourceKind": "url", "sourceUri": "{Add yours url}", "refresh": "true" } }
Endpoint : https://{Endpoint}/language/query-knowledgebases/projects/{projectName}/sources?api-version=2021-10-01
and in headers I'm passing Ocp-Apim-Subscription-Key
-
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-03-30T08:28:56.29+00:00 @VasaviLankipalle-MSFT I resolved this bad request issues by passing the request as an array for patch call. But now it's getting 202 and in response I'm getting empty string.
it's not getting updated in the portal. I appreciate if you could help!!!.
-
VasaviLankipalle-MSFT 17,476 Reputation points
2023-03-30T18:34:04.8933333+00:00 Hi @KuppiReddygariSamthrushaReddy-3789 , thanks for sharing this with us.
You can use the
Operation-Location
header to check the status of the request to the URL specified in the header. To retrieve the response headers in your curl command, you can use the-i
option.here is an example query from the documentation.
curl -X PATCH -H "Ocp-Apim-Subscription-Key: {API-KEY}" -H "Content-Type: application/json" -d '[ { "op": "add", "value": { "displayName": "source5", "sourceKind": "url", "sourceUri": "https://download.microsoft.com/download/7/B/1/7B10C82E-F520-4080-8516-5CF0D803EEE0/surface-book-user-guide-EN.pdf", "sourceContentStructureKind": "semistructured" } } ]' -i 'https://{ENDPOINT}.api.cognitive.microsoft.com/language/query-knowledgebases/projects/{PROJECT-NAME}/sources?api-version=2021-10-01'
This
-i
option tells curl to include the response headers in the output. Without this option, the response to the command would only include the response body, which may be empty if the operation is still in progress. -
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-03-31T03:06:53.3033333+00:00 But I'm calling that api in c#. How can I add that -i option in C#
-
My KineMaster 0 Reputation points
2023-03-31T03:31:16.0533333+00:00 The code you provided seems to contain a syntax error in the JSON payload. The double quotes used in the JSON string are not escaped properly. Try modifying the JSON payload as shown below, which includes proper escape sequences for the double quotes: arduino Copy code request.Content = new StringContent(@"[ { ""op"": ""replace"", ""value"": { ""displayName"": ""source5"", ""sourceKind"": ""url"", ""sourceUri"": ""https://download.microsoft.com/download/7/B/1/7B10C82E-F520-4080-8516-5CF0D803EEE0/surface-book-user-guide-EN.pdf"", ""refresh"": ""true"" } } ]"); Also, ensure that you have replaced the placeholders {ENDPOINT}, {PROJECT-NAME}, and {API-KEY} with their actual values in the URL and request headers. Additionally, make sure that the URL for the PDF file is valid and accessible.
-
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-03-31T03:42:24.08+00:00 I'm not facing any issue with the payload. I created a model in my code with those properties and I'm serializing them and passing that as a request. object. Only issue I'm facing is that -i optino in curl. How can I add that option in c#
-
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-03-31T04:18:52.3066667+00:00 I tried GET call with operation-location, I'm getting below response.
But It's not getting updated in cognitive service portal. It's still having the old url.
-
romungi-MSFT 46,751 Reputation points • Microsoft Employee
2023-03-31T04:21:31.9233333+00:00 @KuppiReddygariSamthrushaReddy-3789 The operation location URL should be used with a GET request. With C# you can use the HTTP client library to just run a GET request on the URI. The status of the job will be returned as JSON which you can read to check if the job status is complete. The example for curl is a sample to demonstrate the functionality.
As seen in the reference of the API, the operation-location header gives the location to get the status of the job i.e the update sources request that was executed.
If you are using C# it is easier to just use the QuestionAnswering SDK for .NET to handle this operation. Please see this page for snippet of UpdateSources()
-
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-03-31T04:24:36.58+00:00 add and delete operations are working only replace operation is not working after getting succeeded to.
-
romungi-MSFT 46,751 Reputation points • Microsoft Employee
2023-03-31T04:55:53.6033333+00:00 Based on above conversation, the replace operation did not use the
source
field. This could be the reason for the operation to not replace or refresh. Please see reference from this page for all operations.As per reference:
{ "op": "replace", "value": { "displayName": "source1", "sourceUri": "https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/overview/overview", "sourceKind": "url", "refresh": true, "source": "https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/overview/overview" }
As per the request that was used from above conversation:
{ "op": "replace", "value": { "displayName": "source5", "sourceKind": "url", "sourceUri": "{Add yours url}", "refresh": "true" } }
-
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-03-31T06:10:16.0166667+00:00 I added source to the request object. now it's working. Thanks @romungi-MSFT & @VasaviLankipalle-MSFT for your support
-
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-04-03T12:15:39.9233333+00:00 Hello @romungi-MSFT and @VasaviLankipalle-MSFT , Do we have any option to delete all the sources in the kb without specifying the source?
-
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-04-04T08:25:32.8733333+00:00 @romungi-MSFT and @VasaviLankipalle-MSFT It's adding duplicate answers in kb with replace operation.
It should update the existing answer right? Why is it creating duplicates?
-
romungi-MSFT 46,751 Reputation points • Microsoft Employee
2023-04-04T08:35:56.9466667+00:00 @KuppiReddygariSamthrushaReddy-3789 I believe you are replacing the source rather than the question and pair. Does your source point to the new location that you passed in the replace request? I think if you need to delete the duplicates you can use the filter and delete the pairs which are not required. Thanks!!
-
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-04-04T08:38:41.79+00:00 @romungi-MSFT how can I identify and delete them from code with question answering sdk.
-
romungi-MSFT 46,751 Reputation points • Microsoft Employee
2023-04-04T09:13:30.9866667+00:00 @KuppiReddygariSamthrushaReddy-3789 If you want to update a particular question answer pair then you will have to use Update question and answer pairs API. It's equivalent with the SDK should be UpdateQnas. You can also use the export and import API to export the project and its assets to a file and remove the pairs that are not required and import the project and its assets with a different project name and proceed forward. If you are testing out the functionality and have lot of edits to your sources then this can be a easier way to create a new project when you finalize what should be the KB and deploy it as an endpoint. I hope this helps!!
-
KuppiReddygariSamthrushaReddy-3789 66 Reputation points
2023-05-22T10:36:14.2233333+00:00 @romungi-MSFT I followed the same approach as per the documentation. I created a automation to maintain different kb's so I want to perform these operations sequentially.
- Get the sources
- Delete those sources
- Add new sources
- Deploy the project.
It's working fine until now. But today I'm facing a strange issue mentioned in the below thread.
Can you please tell me what is the issue? Why am I getting the old kb data ??
Sign in to comment