System Message not working on Azure Openai client library

Vogel, Sabine (Sabine) 0

The system message with same instructions works for testing in Azure OpenAI Studio but not with Azure Openai client library for Javascript. System message is ignored here. I use model version GTP-3.5 Turbo 1106.

This question was already asked in October. The response pointed out that this is a bug.

Now my question: Has this error been fixed by now?

AshokPeddakotla-MSFT 27,401 Reputation points

2024-04-16T13:10:14.92+00:00

Vogel, Sabine (Sabine) Did you get a chance to see my suggestions?

Do let me know if that helps or have any further queries.

1 answer

AshokPeddakotla-MSFT 27,401 Reputation points

2024-04-10T05:56:29.4933333+00:00

Vogel, Sabine (Sabine) Greetings!

The system message with same instructions works for testing in Azure OpenAI Studio but not with Azure Openai client library for Javascript. System message is ignored here. I use model version GTP-3.5 Turbo 1106. This question was already asked in October. The response pointed out that this is a bug. Now my question: Has this error been fixed by now?

Azure OpenAI's support for roleInformation and other extended On Your Data features was added in the 2023-12-01-preview service API version and thus wasn't yet present in the beta.5 version of the Java SDK, which was released based on the 2023-09-01-preview service API version.

roleInformation is among the many additions for On Your Data support in this version, as can be seen in the public source here.

Also, See Azure OpenAI client library for JavaScript - version 1.0.0-beta.12

Please try with the latest version and let me know if you are facing any issues.
Please sign in to rate this answer.
Vogel, Sabine (Sabine) 0 Reputation points

2024-04-16T13:25:40.7+00:00

Hi Ashok,

thanks a lot for your answer. Using the roleInformation for the system message works in general. But the instructions in the system message are sometimes ignored. Is this a known problem?

AshokPeddakotla-MSFT 27,401 Reputation points

2024-04-18T10:06:06.36+00:00

This should be resolved by the latest updates. Could you confirm if you are still seeing the issue?

If yes, please share more context along with screenshots for further help.

Vogel, Sabine (Sabine) 0 Reputation points

2024-04-18T11:19:27.29+00:00

I have tested now only in playground and uploaded only one document with parking information but I'm just asking questions about the name (instruction "Your name is Ted." in system message). Sometimes this question is answered correctly and sometimes the information is not found.

AshokPeddakotla-MSFT 27,401 Reputation points

2024-04-23T10:53:35.8233333+00:00

Vogel, Sabine (Sabine) Apologies for the delayed response.

As mentioned here, The system message is used in addition to an internal base prompt to provide the experience. To support this, we truncate the system message after a specific number of tokens to ensure the model can answer questions using your data. If you are defining extra behavior on top of the default experience, ensure that your system prompt is detailed and explains the exact expected customization.

I just tried with GPT-35-turbo model and able to see the expected results.

Please see below for more clarity.

Do let me know if that helps or have any further queries.

Vogel, Sabine (Sabine) 0 Reputation points

2024-04-24T10:06:38.1433333+00:00

If I use a data source and additionally write in the system message what the AI assistant is called, then the questions for the name are usually not answered. If no documents are used and only the system message is tested it works.
Is the combination between uploaded documents and instructions in system message not possible?
I use gpt-35-turbo, too.

AshokPeddakotla-MSFT 27,401 Reputation points

2024-04-25T02:14:23.95+00:00

Vogel, Sabine (Sabine)

Is the combination between uploaded documents and instructions in system message not possible? I use gpt-35-turbo, too.

I have added a data source earlier. System message is an addition to the internal base prompt.

As mentioned earlier, the system message is truncated after a specific number of tokens to ensure the model can answer questions using your data. If you are defining extra behavior on top of the default experience, ensure that your system prompt is detailed and explains the exact expected customization.

I would suggest you, clear the chat and try with new system message once.
Sign in to comment