Function call with Azure OpenAI gpt-4o returns 400 error although the messages are valid

Question

Hi support team,

I am testing the new Azure Open AI gpt-4o model but I get the error with function call even though my input messages are correct.

Here are the code:

from dotenv import load_dotenv
 
load_dotenv()
 
from openai import OpenAI, AzureOpenAI
# client = OpenAI()
client = AzureOpenAI(
    azure_endpoint='https://xxx.openai.azure.com/',    
    api_version="2024-02-01",
    api_key='xxx'
)

msg=[
    {
        "role": "user", 
        "content": [
            {"type": "text", "text": "What is in the image?"}, 
            {
                "type": "image_url", 
                "image_url": {
                    "url": "https://images.freeimages.com/images/large-previews/c04/puppy-1367856.jpg"
                }
            }
        ]
     }, 
     {"role": "assistant", "content": "There is a small dog in the image"}, 
     {"role": "user", "content": [{"type": "text", "text": "What is today?"}]}, 
     {"role": "assistant", "content": "", "tool_calls": [{"id": "call_rkBDeJsY9mCnzFchblUV2ZHt", "function": {"arguments": "{}", "name": "get_current_date"}, "type": "function"}]}, 
     {"tool_call_id": "call_rkBDeJsY9mCnzFchblUV2ZHt", "role": "tool", "name": "get_current_date", "content": "2024-05-21 19:52:24.775703"}
]

completion = client.chat.completions.create(
    model='gpt-4o',
    messages=msg,           
    temperature=0
)
choice = completion.choices[0].message
print(choice)

And this is the error message

openai.BadRequestError: Error code: 400 - {'error': {'message': "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_rkBDeJsY9mCnzFchblUV2ZHt", 'type': 'invalid_request_error', 'param': 'messages', 'code': None}}

Notes:

• The error only occurs when I include the image https://images.freeimages.com/images/large-previews/c04/puppy-1367856.jpg in the first question
• When I use OpenAI service instead of AzureOpenAI, the code is running perfectly. I believe that is the issue with AzureOpenAI client. Could you please help investigate this issue? Thanks.