Azure OpenAI "There was an error generating a response." or "An error occurred."

Marcia 15 Reputation points

I have deployed an Azure OpenAI chat using both the models GPT-35-Turbo and GPT-4. The chat seems to be working for a while, but at a certain point, guaranteed, I get the following errors in the chat:


There was an error generating a response. Chat history can't be saved at this time. If the problem persists, please contact the site administrator.

An error occurred. Answers can't be saved at this time. If the problem persists, please contact the site administrator.

Azure OpenAI errors

After I get this error, the chat is broken and I can no longer proceed with the conversation. As soon as I start a new chat/conversation, there are no errors until that conversation also starts to get errors.

This has been very frustrating as a history of a conversation can no longer be used and I have to start over with a chat. As well, the new chat will give errors after a certain time as well.

I have used West Europe and Sweden as the region, but both give the errors.

Please help as we are paying for Azure OpenAI but without it actually working correctly.

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,639 questions
Azure AI services
Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
2,643 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Cumai 0 Reputation points

    Finally found the issue and solve it, it is working perfectly fine for me now, it is an issue with answer streaming.

    If the answer text streaming is not important for you, you can solve it by adding :

    "SHOULD_STREAM = False "

    to your app environment variables, or you can directly edit the and comment the existing condition set to read from the environment variables :

    SHOULD_STREAM = False 
    ## if AZURE_OPENAI_STREAM.lower() == "true" else False ##
    0 comments No comments