Fine-tuning Azure OpenAI Model GPT-3.5 for Custom Query Language

Question

I want to fine-tune a model for converting text to a custom query language and am facing the following challenges:

1. I am considering using GPT-3.5 Turbo, but are there better alternatives?
2. I have a limited input dataset of 50 samples for each fields which can be used for fine tuning.
3. There are approximately 1,000 fields in total, but initially, I want to launch with 100 fields. The remaining fields will be added gradually in sprints (e.g., Sprint 1 has 100 fields, Sprint 2 has another 100, etc.). What is the best approach for implementation and fine-tuning in this scenario?
4. Since I cannot include all 1,000 fields in the prompt, what is a better way to train the model to accommodate this?
5. Is it possible to establish a feedback loop for the fine-tuned GPT-3.5 model?

Thanks in advance for the help!

Answer 1

Hello Dhaval Kansagra,

Thank you for reaching out on the Microsoft Q&A .

1. GPT-3.5 Turbo is a solid option for this. If you want something even more advanced, GPT-4 is a choice, though it may be pricier and a bit more than you need.
2. With only 50 samples per field, you could get more mileage from your data by using methods like paraphrasing to expand your dataset.
3. Start with an initial set of 100 fields, and fine-tune with each sprint. This lets you add fields incrementally without overwhelming the model and allows you to test as you go.
4. Instead of putting all fields in each prompt, use few-shot examples to pull in only the relevant fields. This keeps prompts manageable and focused.
5. With reinforcement learning or active learning, you can also prioritize feedback on the most useful examples.

I hope this helps! If you have any further questions, feel free to ask.

If the information is useful, please accept the answer and upvote it to assist other community members.

Answer 2

Hi @Dhaval Kansagra,

Thank you for reaching out to Microsoft Q&A forum!

Here are the following responses for you queries:

I am considering using GPT-3.5 Turbo, but are there better alternatives?

GPT-3.5 Turbo is a good choice for structured tasks like query generation. However, if available, GPT-4 model may offer improved generalization and accuracy on limited data. Start with GPT-3.5 to gauge performance and consider GPT-4-turbo for scalability.

I have a limited input dataset of 50 samples for each fields which can be used for fine tuning.

If you have a limited dataset of 50 samples per field, focus on carefully selecting representative examples that cover different variations of each field. Fine-tune the model on this curated set to help it learn general patterns. Additionally, use few-shot prompting during inference to guide the model in generating responses based on the few examples it has seen, allowing it to adapt more effectively to new inputs.

There are approximately 1,000 fields in total, but initially, I want to launch with 100 fields. The remaining fields will be added gradually in sprints (e.g., Sprint 1 has 100 fields, Sprint 2 has another 100, etc.). What is the best approach for implementation and fine-tuning in this scenario?

Start by fine-tuning the model on the first 100 fields, ensuring those fields are well-represented with diverse examples. For each subsequent sprint, fine-tune the model on the next batch of 100 fields, while leveraging the previous fine-tuned model as the base. This incremental fine-tuning approach allows the model to gradually adapt to new fields without forgetting the previous ones, ensuring scalability as you add more fields over time.

Since I cannot include all 1,000 fields in the prompt, what is a better way to train the model to accommodate this?

To avoid prompt overload, set structured response rules that the model references during output. Alternatively, consider using an external lookup or database to help manage field-specific information without listing each field in the prompt.

Is it possible to establish a feedback loop for the fine-tuned GPT-3.5 model?

While a direct feedback loop for fine-tuned GPT-3.5 is not possible within the model itself, you can implement an indirect feedback system. Collect user ratings or comments on the model's outputs and manually analyze them to identify common errors or areas for improvement. This feedback can then be used to curate a new dataset or refine existing training examples for future fine-tuning sessions. Regularly updating your training set based on this analysis can help improve the model over time without real-time feedback integration.

I hope you understand. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful.