Azure OpenAI GPT-4-Vision streaming bug

Question

Good day, experts.

I have a problem when I use GPT-4-Vision model deployed in Australia East using Azure OpenAI service. When I use stream option enabled like in the code below:

            response = chat_client.chat.completions.create(
                            model=engine,
                            messages=gpt_messages, 
                            temperature=temperature,
                            stream=True,
                            max_tokens = output_token_limits[engine]
                            )

The GPT-4-vision returns "Sorry, I can't help with that request." most of the time, like 95% and occasionally it does work.

However, should I switch off streaming, it works fine and consistent.

Has anyone observed that and found a solution?

Thanks in advance.

Answer 1

@Alexander Novikov I believe these two versions support the vision model currently, if you are using the first API version(2023-12-01-preview) and seen this error, could you try the second one (2024-02-15-preview) to check if it is consistent?

• 2023-12-01-preview Swagger spec
• 2024-02-15-previewSwagger spec

Also, in the messages input you can try to use the image_url first and then the text. Something similar to below and check if the behavior improves?

"role": "user",

            "content": [

                          {

                          {

                              "type": "image_url",

                              "image_url": {

                        "url": <your_url>

                    },

                              "type": "text",

                              "text": "Describe this picture:"

                          }

                }

           ]

        }