Why does GPT-4-O in azure openAI randomly return inappropriate content errors when the content is appropriate?

AMROUN Lysa 361

good morning,

i'm using gpt4-o with azure openai in the US2 region, i've observed that it often returns inappropriate content errors knowing that my content is not at all inappropriate, the same request can be considered as inappropriate content at time t, and normal content just after (it's random). i've modified the “content filter” at different levels but without success, knowing that i don't encounter this problem with gpt4-turbo and gpt3.5-turbo.

ps: I use the french chat, I don't know if it depends on that too.

could you tell me how to limit this kind of error when it's not about inappropriate content?

thank you in advance for your help

lysa

navba-MSFT 25,075 Reputation points Microsoft Employee

2024-05-31T10:10:21.1666667+00:00
@AMROUN Lysa Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

.

Please let me know how are you accessing the Azure OpenAi service through REST API / SDK or Open Ai Studio ?

Please clarify what does inappropriate content mean.

Are you getting any error / exception message ?

Please share the screenshot with the returned content VS expected content.

Is this something which started recently ? or was it working earlier and returning the right content ?

Is the issue intermittent or occurs all the time ?

May I know which region is the resource deployed to ?

Are you using Add your own data feature ?

.

Awaiting your reply.
AMROUN Lysa 361 Reputation points

2024-05-31T14:49:20.47+00:00
hi @navba-MSFT

thanks for your response,

1.      Please let me know how are you accessing the Azure OpenAi service through REST API / SDK or Open Ai Studio ?

I access azure OpenAI in REST API/SDK of azure in python , like this :

response = openai.chat.completions.create( model=os.getenv("AZURE_OPENAI_DEPLOYMENT_ID_US"), messages=self.conversation ) assistant_response = response.choices[0].message.content self.conversation.append({"role": "assistant", "content": assistant_response})

2.      Please clarify what does inappropriate content mean.

**=> inapproriated content is any content prohibited by azure openai such as sexual content, violent content, ...**

3.      Please share the screenshot with the returned content VS expected content.

**=> This is the chat response :**

Error code: 400 - {'error': {'message': "The response was filtered due to the prompt triggering Azure OpenAI's content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766", 'type': None, 'param': 'prompt', 'code': 'content_filter', 'status': 400, 'innererror': {'code': 'ResponsibleAIPolicyViolation', 'content_filter_result': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': True, 'severity': 'high'}, 'violence': {'filtered': False, 'severity': 'safe'}}}}}

   For this request in french : « Aide moi a rédiger un mail pour un client afin d'obtenir un rendez vous client TECUMSEF > in english it’s : « Help me to write a mail for a customer in order to get a customer appointment TECUMSEF »

4.      Is this something which started recently ? or was it working earlier and returning the right content ?

=> I only started using gpt4-o from azure oenai recently (1 week) and it happened to me from the first application, but with gpt3.5-turbo or gpt4-turbo it never happened to me with normal content, unlike with gpt4-o.

5.      Is the issue intermittent or occurs all the time ?

=> It’s intermittent and random

6.      Are you using Add your own data feature ?

=> No , i use it only to generate text with a system instruction and user prompt without my own data.

thank's

Lysa
navba-MSFT 25,075 Reputation points Microsoft Employee

2024-05-31T16:27:42.2833333+00:00

@AMROUN Lysa Thanks for getting back and sharing the details.

.

While I used the same text from chat playground for my GPT4o model it worked just fine. See below:

Could you please check if you are encountering the same issue from Azure Open AI Studio chat playground ?

The default content filtering configuration is set to filter at the medium severity threshold for all four content harms categories for both prompts and completions. That means that content that is detected at severity level medium or high is filtered, while content detected at severity level low or safe is not filtered by the content filters.

Could you please check the Content Filters and its severity set for the sexual filter ? Could you please set it to low and check ?

Awaiting your reply.
Peder Holdgaard Pedersen 60 Reputation points

2024-06-03T08:47:52.74+00:00

This also happens via the 'direct' OpenAI API and the ChatGPT web app. I suspect it is a bug/issue related to the GPT-4o model and not actually moderation.

I do not have access to GPT-4o via Azure, but would not be surprised if this is the same underlying issue, based on the lack of moderation-relevant content, which is also the observation I have made in the other environments that use GPT-4o. Perhaps it is triggered when the model responds with a message in a modality that aren't currently available? (ie audio/image output?)

To reproduce it, one can include a number of image messages (3-4) and then increase the (text) message count gradually in the request.
navba-MSFT 25,075 Reputation points Microsoft Employee

2024-06-03T09:40:32.23+00:00

@AMROUN Lysa I have sent you a private message asking for more details about your resource, so that I can look at the backend logs for the cause of this issue.

Awaiting your reply.
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Ashutosh Tripathi 0 Reputation points

2024-10-01T14:04:44.7866667+00:00

Hey @AMROUN Lysa
Were you able to find a solution?
I am facing similar issue and tried several options but none of them worked.
I tried using GPT-4 Turbo as well as created a bit relaxed content filter, but still no luck.

One solution was to retry the request that is resulting in NULL response. That was working fine after 3rd retry. I would rather prefer a working solution then to retry on these random failures.

Share via

Why does GPT-4-O in azure openAI randomly return inappropriate content errors when the content is appropriate?

Your answer