Self train OpenAI model using response and feedback

Question

I'm creating a custom model using a based model with my own dataset. This model is accessing through REST API to answer customer questions.

My concern here is I want to get the customer's feedback for the answer they are receiving though the model and send their feedback back to model. And with that feedback i want to self-train the model/ improve model performance AUTOMATICALLY.

Answer 1

Hello @SajithHettiarachchi-0552

Thanks for reaching out to us, there is currently no automatically way to continue fine-tuning the model.

You need to build a process by yourself and include the continuing fine-tuning as part of it.

https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/fine-tuning?tabs=turbo%2Cpython-new&pivots=programming-language-python#continuous-fine-tuning

An example of this process is as below -

Collect Feedback

Implement Feedback Collection

• Integrate Feedback Mechanism: Include a feedback option in your application where users can rate or provide comments on the model’s responses.
• Store Feedback: Collect and store feedback in a structured format. For example, you could use a database or cloud storage (like Azure SQL Database or Azure Blob Storage) to save feedback data.

Analyze and Prepare Feedback

Data Aggregation

• Collect Responses and Feedback: Gather all interactions, including the original model responses and user feedback.
• Label Data: Depending on your feedback, label the data accordingly. For example, if feedback is positive or negative, or if it provides suggestions for improvement.

Data Cleaning

• Filter Feedback: Remove noisy or irrelevant feedback. Ensure that the feedback is useful for training.
• Format Data: Convert feedback into a format suitable for training, such as pairs of prompts and improved responses.

Prepare Training Data

Data Annotation

• Create Training Examples: Prepare training examples from feedback, such as prompt-response pairs with corrected or improved responses.
• Organize Data: Structure the data to be used for fine-tuning. Ensure consistency and quality.

Automate Model Training

Use OpenAI’s Fine-Tuning API (or equivalent in Azure OpenAI)

• Prepare Training Scripts: Write scripts to prepare your data for fine-tuning.
• Fine-Tuning Process:
  • Create a Dataset: Upload your training data to the OpenAI or Azure OpenAI fine-tuning platform.
    • Initiate Fine-Tuning: Use the fine-tuning API to train your model on the new data.
      • Monitor Training: Track the training process to ensure it completes successfully.

Automate Training Pipelines

• Create Automation Scripts: Develop scripts or workflows to automate the process of collecting feedback, preparing data, and triggering fine-tuning.
• Set Up Scheduling: Use Azure services like Azure Functions or Logic Apps to schedule and automate these processes based on feedback accumulation.

Deploy and Test Updated Model

Deploy the Fine-Tuned Model

• Deployment: Deploy the newly trained model to your production environment.
• Testing: Perform thorough testing to ensure the updated model performs as expected.

Continuous Monitoring

• Monitor Performance: Continuously monitor the performance of the model to ensure improvements.
• Adjust Training: Based on performance, adjust your training process and data preparation as needed.

Please adjust this process accordingly. I hope this helps.

Regards,

Yutong

-Please kindly accept the answer if you feel helpful to support the community, thanks a lot.