JSON Array output from Azure OpenAI via API

Question

I am calling Azure OpenAI via the API. My input is a sentence. The sentence contains either 1 part or 2 parts. My prompt handles whether it has 1 part or 2 parts and instructs it to output a json array. If it has 2 parts, there will be an element in the json output for each part. I have verified that my prompt works in the Azure AI Foundry Chat Playground with no issues whether 1 part or 2 parts.

However, when I use the same prompt via the Azure OpenAI API, it only returns the 1st part when 2 parts are present in the input.

Can someone help me understand why this is happening? Is this a limitation of the API?

Answer 1

Hi Stephen!

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer.

Ask: Issue on Azure OpenAI on getting respective Json output for 2-part sentence

Solution: The issue was resolved by adopting structured output response.

If I missed anything please let me know and I'd be happy to add it to this answer, or feel free to comment below with any additional information.

If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

Thank You.

Answer 2

Thanks for the info. I actually just solved my issue. I kept the system prompt that was working in Azure AI Foundry Chat Playground. I used the updated documentation to specify a structured output (the json array) in the API. With the structured output it worked with 1 part or 2 parts.

The structured output docs are at: https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/structured-outputs?tabs=python-secure%2Cdotnet-entra-id&pivots=programming-language-rest

Thanks.