How to implement 'Regenerate' button on ChatGPT 3.5 model?

Question

How to implement 'Regenerate' button on ChatGPT 3.5 model?

Karuna Tiwari 100

Hi,
We have created a BYOD Chatbot using ChatGPT 3.5 16K model. Now I want to add a button in frontend named 'Regenerate' which the user clicks when he/she is not happy with the response generated and wants to run the query again.

What code should be written in the backend to enable this feature?

navba-MSFT 27,485 Reputation points Microsoft Employee

2024-03-13T04:43:29.8533333+00:00
@Karuna Tiwari Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

To implement a ‘Regenerate’ button in your chatbot, you would need to make another API call to the Azure Open AI model endpoint with the same prompt when the ‘Regenerate’ button is clicked.

Here’s a simplified example of how you could implement this in Python using Flask and the OpenAI API:

from flask import Flask, request import os from openai import AzureOpenAI app = Flask(__name__) client = AzureOpenAI( api_key=os.getenv("AZURE_OPENAI_API_KEY"), api_version="2023-12-01-preview", azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT") ) deployment_name='REPLACE_WITH_YOUR_DEPLOYMENT_NAME' #This will correspond to the custom name you chose for your deployment when you deployed a model. Use a gpt-35-turbo-instruct deployment. @app.route('/chat', methods=['POST']) def chat(): message = request.json['message'] response = client.completions.create(model=deployment_name, prompt=message, max_tokens=10) return response.choices[0].text if __name__ == '__main__': app.run()

In this code, when a POST request is made to the ‘/chat’ endpoint with a message, the chatbot generates a response using the Azure OpenAI model. If the user is not satisfied with the response and clicks the ‘Regenerate’ button, your frontend should make another POST request to the ‘/chat’ endpoint with the same message, and the chatbot will generate a new response.

Please replace 'REPLACE_WITH_YOUR_DEPLOYMENT_NAME' with your actual deployment name. Also, remember to handle exceptions and edge cases according to your application’s needs. This is a simplified example and might need modifications based on your application’s architecture and requirements.

More info about the Python Azure Open AI code is available here.

Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help.
Karuna Tiwari 100 Reputation points

2024-03-14T05:19:15.0666667+00:00

Thanks for your reply navba-MSFT.

Does that mean that regenerate function works similar to the 'Clear chat' option where we delete all the previous context and start afresh?
navba-MSFT 27,485 Reputation points Microsoft Employee

2024-03-14T05:26:04.5266667+00:00

@Karuna Tiwari Thanks for getting back. The Regenerate functionality option shared above will only query the open AI endpoint and generate a new response if the user is not happy with the previous response.

It doesn't clear the previous chat (conversation history) in the existing chat window. For that, you need to add custom logic / custom code in your application to clear it. Hope this answers.
Karuna Tiwari 100 Reputation points

2024-03-14T05:30:45.31+00:00

@navba-MSFT Thanks for your prompt reply.

I understand that the Regenerate button helps to generate a new response in case the user is not happy with the previous response.
But what if the temperature is set to 0? Will it not generate the same response again?
Karuna Tiwari 100 Reputation points

2024-03-14T06:03:33.3966667+00:00
This definite helps @navba-MSFT . Thanks.

One last Ques - How is the Regenerate functionally implemented in the current GPT 3.5.

It asks the same query again, keeping all the context (prompt + past history) intact.

It asks the same query again, deleting the context (Chat History is not deleted from the screen). Only the query and prompt are passed to the model.

Is it option 1 or 2?
navba-MSFT 27,485 Reputation points Microsoft Employee

2024-03-14T06:08:59.0766667+00:00

@Karuna Tiwari Thanks for getting back. You are using ChatGPT from OpenAI ( and not Microsoft Azure OpenAI ). So I may not be able to provide you a direct answer.

In the content of Azure Open AI (with model GPT3.5), The Regenerate functionality can be implemented in either way, depending on the specific requirements of your application. Both options have their own implications:

Option 1: If you choose to keep all the context (prompt + past history) intact, the model will have more information to generate a response. This can be beneficial if the context is important for generating a meaningful response. However, it might also limit the variability of the responses, as the model will try to maintain coherence with the existing conversation history.

Option 2: If you choose to delete the context and only pass the query and prompt to the model, you might get more diverse responses. This is because the model is not constrained by the past conversation history. However, the responses might not always be relevant or coherent with the previous conversation.

In the end, the choice between Option 1 and Option 2 depends on the specific needs of your application. You might want to experiment with both options and see which one works best for your use case. Hope this helps.
Karuna Tiwari 100 Reputation points

2024-03-14T06:12:03.16+00:00

Makes sense. Thanks for your time.
navba-MSFT 27,485 Reputation points Microsoft Employee

2024-03-14T06:26:34.7033333+00:00

@Karuna Tiwari Thanks for your reply. For the below answer, Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Accepted answer

0 additional answers

Your answer

Karuna Tiwari 100 Reputation points

2024-03-14T05:19:15.0666667+00:00

Thanks for your reply navba-MSFT.

Does that mean that regenerate function works similar to the 'Clear chat' option where we delete all the previous context and start afresh?
navba-MSFT 27,485 Reputation points Microsoft Employee

2024-03-14T05:26:04.5266667+00:00

@Karuna Tiwari Thanks for getting back. The Regenerate functionality option shared above will only query the open AI endpoint and generate a new response if the user is not happy with the previous response.

It doesn't clear the previous chat (conversation history) in the existing chat window. For that, you need to add custom logic / custom code in your application to clear it. Hope this answers.
Karuna Tiwari 100 Reputation points

2024-03-14T05:30:45.31+00:00

@navba-MSFT Thanks for your prompt reply.

I understand that the Regenerate button helps to generate a new response in case the user is not happy with the previous response.
But what if the temperature is set to 0? Will it not generate the same response again?
Karuna Tiwari 100 Reputation points

2024-03-14T06:03:33.3966667+00:00

This definite helps @navba-MSFT . Thanks.

One last Ques - How is the Regenerate functionally implemented in the current GPT 3.5.

It asks the same query again, keeping all the context (prompt + past history) intact.

It asks the same query again, deleting the context (Chat History is not deleted from the screen). Only the query and prompt are passed to the model.

Is it option 1 or 2?
navba-MSFT 27,485 Reputation points Microsoft Employee

2024-03-14T06:08:59.0766667+00:00

@Karuna Tiwari Thanks for getting back. You are using ChatGPT from OpenAI ( and not Microsoft Azure OpenAI ). So I may not be able to provide you a direct answer.

In the content of Azure Open AI (with model GPT3.5), The Regenerate functionality can be implemented in either way, depending on the specific requirements of your application. Both options have their own implications:

Option 1: If you choose to keep all the context (prompt + past history) intact, the model will have more information to generate a response. This can be beneficial if the context is important for generating a meaningful response. However, it might also limit the variability of the responses, as the model will try to maintain coherence with the existing conversation history.

Option 2: If you choose to delete the context and only pass the query and prompt to the model, you might get more diverse responses. This is because the model is not constrained by the past conversation history. However, the responses might not always be relevant or coherent with the previous conversation.

In the end, the choice between Option 1 and Option 2 depends on the specific needs of your application. You might want to experiment with both options and see which one works best for your use case. Hope this helps.
Karuna Tiwari 100 Reputation points

2024-03-14T06:12:03.16+00:00

Makes sense. Thanks for your time.
navba-MSFT 27,485 Reputation points Microsoft Employee

2024-03-14T06:26:34.7033333+00:00

@Karuna Tiwari Thanks for your reply. For the below answer, Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Answer 1

@Karuna Tiwari Thanks for your question. Regarding the temperature parameter value ( default is 1 ). This parameter explains, what sampling temperature to use, between 0 and 2. Higher values mean the model takes more risks. Try 0.9 for more creative applications, and 0 (argmax sampling) for ones with a well-defined answer.

More info ***here ***and here.

To answer your question, Yes, you’re correct. If the temperature is set to 0, the model becomes completely deterministic and will generate the same response every time for a given input. This is because a temperature of 0 makes the model always choose the most likely next word when generating text.

Please note, it’s important to note that even with a temperature of 0, there might be slight variations in the responses due to the inherent randomness in the model’s architecture. But these variations are usually minimal and the overall intent of the response remains the same.

Recommendation:

So, if you’re implementing a ‘Regenerate’ button and you want to see different responses, you might want to consider using a higher temperature value. This will introduce more randomness into the generated responses, giving you a wider variety of potential outputs for the same prompt. But remember, a higher temperature also means the responses might be less focused and coherent. It’s all about finding the right balance for your specific use case.

Hope this answers.

navba-MSFT 27,485 Reputation points Microsoft Employee

2024-03-14T06:27:13.8133333+00:00

@Karuna Tiwari For the above answer, Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Share via

How to implement 'Regenerate' button on ChatGPT 3.5 model?

0 additional answers

Your answer