Azure Open AI Service API does not consider own data in responses

Question

Dear all,

I was able to deploy a gpt-35-turbo model via https://oai.azure.com/ with custom data and in the chat playground it works perfectly. If I do an Azure Web App deploy, it also works perfectly.

I would like to embed it into a custom website (Power Agents is not working in my region) via API calls based on this documentation: https://learn.microsoft.com/en-gb/azure/ai-services/openai/chatgpt-quickstart?tabs=command-line&pivots=programming-language-javascript

I completed all the pre-requisites, and everything works.

I try to get the info from here:

Which shows me the url for the API and the KEY:

When I try to call this endpoint, it returns a complete hallucination for a given question - as if it would not take the custom data into account.

This is really strange, as the chat playground takes the custom data into account, and replies accordingly.

Anyone has any idea why is this happening?

Answer 1

The solution for these problem is to use the Example cURL command from this documentation: https://learn.microsoft.com/en-us/azure/ai-services/openai/use-your-data-quickstart?tabs=command-line&pivots=rest-api

I use Node 18 with Axios. Works perfectly.

Additional explanation: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data#system-message

Basically this way you can create an API response in similar quality the https://oai.azure.com/ web app deploy creates, but use with your custom frontend.

Use this basic deployment to validate your assistant answers.

Answer 2

@Balazs Nemeth Thanks for the details, It's known issue when use the completion endpoint API with out the chat ML. we will update once the issue is fixed for completion endpoint API to use add your own data.

you are not using ChatML, thus you are not using the model as it is intended. Results can vary.