Hi. It appears our AI Assistants deployed on gpt-4o are losing their chat history & context. We feed the AI Assistant context specific data about the user on 1st call. Then we're having to re-feed the entire chat history + context on each call to the AI Assistants. Is that normal? Isn't it enough to just call the thread ID and user continues their chat "where they left off"? Does the thread ID save context & user information provided OnInit()? To refresh this context information periodically (as applicable), do we simply pass new (incremental) context or full text back each time? Is every conversation getting appended to a unique thread ID (1 thread ID for 1 user)? Can we simply retriev the thread on each call and the entire chat history will be fetched? Where is the chat history saved? Thnx.

AI Assistants deployed on gpt-4o base model are forgetting chat history & context.

GenixPRO 36

Hi.

It appears our AI Assistants deployed on gpt-4o are losing their chat history & context. We feed the AI Assistant context specific data about the user on 1st call. Then we're having to re-feed the entire chat history + context on each call to the AI Assistants. Is that normal? Isn't it enough to just call the thread ID and user continues their chat "where they left off"?
Does the thread ID save context & user information provided OnInit()? To refresh this context information periodically (as applicable), do we simply pass new (incremental) context or full text back each time?
Is every conversation getting appended to a unique thread ID (1 thread ID for 1 user)? Can we simply retriev the thread on each call and the entire chat history will be fetched? Where is the chat history saved?

Thnx.

AshokPeddakotla-MSFT 34,111 Reputation points

2024-10-16T06:41:07.4566667+00:00

GenixPRO Greetings!

Please see below answers to your queries.

It appears our AI Assistants deployed on gpt-4o are losing their chat history & context. We feed the AI Assistant context specific data about the user on 1st call. Then we're having to re-feed the entire chat history + context on each call to the AI Assistants. Is that normal? Isn't it enough to just call the thread ID and user continues their chat "where they left off"?

AFAIK, You need to include the relevant chat history and any contextual information with each request.

When you chat with a model, providing a history of the chat will help the model return higher quality results. You don't need to include the context property of the assistant messages in your API requests for better response quality. See the API reference documentation for examples.

Does the thread ID save context & user information provided OnInit()? To refresh this context information periodically (as applicable), do we simply pass new (incremental) context or full text back each time?

I think thread ID itself does not save context and user information provided in OnInit().

The thread ID is simply a unique identifier for the conversation thread and can be used to maintain the conversation history. You need to store this information separately and pass it back to the AI assistant on each call.

Is every conversation getting appended to a unique thread ID (1 thread ID for 1 user)? Can we simply retriev the thread on each call and the entire chat history will be fetched? Where is the chat history saved?

You can try List Messages that returns a list of messages for the given thread.

As mentioned earlier, the chat history is not saved by the Azure OpenAI service itself. You need to manage this storage in your application.

Do let me know if that helps or have any further queries.
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
GenixPRO 36 Reputation points

2024-10-17T12:57:26.8233333+00:00

Hi.

If <thread ID can be used to maintain the conversation history> AND we don't delete/purge thread ID -> doesn't that mean the conversation history is maintained forever in the thread? If so, when user switches device -> we can migrate the conversation history to new device using thread ID + List Messages(). Correct?
GenixPRO 36 Reputation points

2024-10-17T13:02:59.6633333+00:00
Hi. Thank you for your detailed reply. To clarify, you mention:

thread ID is simply a unique identifier for the conversation thread and can be used to maintain the conversation history

would it be correct then that when a user switches their device, we can migration the conversation history to new device using thread ID + List Messages function? In that case, one may not need to save message history. only associate thread ID to user profile.

Share via

AI Assistants deployed on gpt-4o base model are forgetting chat history & context.

Your answer