Azure OpenAI - GPT-4o Structured Response Format triggers Content Filtering

Question

Good day,

We have discovered what I believe is a bug in the content filtering functionality of the Azure OpenAI resources when enabling and using a Structured Response Format with GPT-4o (2024-08-06).

Using the model to extract a large amount of data from documents with unstructured data (such as contracts and brochures) is something we have been doing for some time now with mixed results. We always enable JSON mode in our requests and include the required JSON Schema in our prompt requests.

This results in the model providing relatively accurate results, but often randomly leaving out information on subsequent prompts. However, that's a different matter.

We also have our own custom filter set up on our GPT-4o deployments with the highest possible tolerances for input and output.

This method of prompting has rarely resulted in our completions being flagged by the content filter.

In an attempt to improve extraction results, we have decided to implement the newer Structured Response Format feature with our requests.

Thus, the JSON schema is no longer included in our prompts but in the request payloads instead. This works relatively well when it works; however, now most of our completions get flagged by the content filter for unrelated issues, such as sexual content.

For example, extracting information from a travel rates brochure will stop halfway through with the content filter triggering an alert for sexual content. We have tried altering our prompts and using other techniques, but the same results occur.

However, processing the exact same information via our older "JSON mode" prompts does not trigger the flag, even though the generated completions are very similar in the end.

I believe this is a bug, or at the very least an issue that needs to be investigated and resolved.