How to implement 'Regenerate' button on ChatGPT 3.5 model?

@Karuna Tiwari Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

To implement a ‘Regenerate’ button in your chatbot, you would need to make another API call to the Azure Open AI model endpoint with the same prompt when the ‘Regenerate’ button is clicked.

Here’s a simplified example of how you could implement this in Python using Flask and the OpenAI API:

from flask import Flask, request
import os
from openai import AzureOpenAI

app = Flask(__name__)

client = AzureOpenAI(
    api_key=os.getenv("AZURE_OPENAI_API_KEY"),  
    api_version="2023-12-01-preview",
    azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
)

deployment_name='REPLACE_WITH_YOUR_DEPLOYMENT_NAME' #This will correspond to the custom name you chose for your deployment when you deployed a model. Use a gpt-35-turbo-instruct deployment.

@app.route('/chat', methods=['POST'])
def chat():
    message = request.json['message']
    response = client.completions.create(model=deployment_name, prompt=message, max_tokens=10)
    return response.choices[0].text

if __name__ == '__main__':
    app.run()

In this code, when a POST request is made to the ‘/chat’ endpoint with a message, the chatbot generates a response using the Azure OpenAI model. If the user is not satisfied with the response and clicks the ‘Regenerate’ button, your frontend should make another POST request to the ‘/chat’ endpoint with the same message, and the chatbot will generate a new response.

Please replace 'REPLACE_WITH_YOUR_DEPLOYMENT_NAME' with your actual deployment name. Also, remember to handle exceptions and edge cases according to your application’s needs. This is a simplified example and might need modifications based on your application’s architecture and requirements.

More info about the Python Azure Open AI code is available here.

Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help.

Thanks for your reply navba-MSFT.

Does that mean that regenerate function works similar to the 'Clear chat' option where we delete all the previous context and start afresh?

@Karuna Tiwari Thanks for getting back. The Regenerate functionality option shared above will only query the open AI endpoint and generate a new response if the user is not happy with the previous response.

It doesn't clear the previous chat (conversation history) in the existing chat window. For that, you need to add custom logic / custom code in your application to clear it. Hope this answers.

@navba-MSFT Thanks for your prompt reply.

I understand that the Regenerate button helps to generate a new response in case the user is not happy with the previous response.
But what if the temperature is set to 0? Will it not generate the same response again?

This definite helps @navba-MSFT . Thanks.

One last Ques - How is the Regenerate functionally implemented in the current GPT 3.5.

1. It asks the same query again, keeping all the context (prompt + past history) intact.
2. It asks the same query again, deleting the context (Chat History is not deleted from the screen). Only the query and prompt are passed to the model.

Is it option 1 or 2?