Correct way to train a custom chatbot using azure open AI?

Question

We are working on creating a chatbot that specializes in a specific domain answers all the questions and shows trends around it (for example - A tech news assistant)
We are using Azure's open AI service to develop this chatbot!
For now, we have done all of this using the RAG methodology
Created a Blob storage and stored all the relevant files in a container

• Used Azure AI search to create an Index that accesses the blob storage
• Created an Azure Open AI Service and created a deployment (GPT4)

There are several issues that we are facing:

• Upon adding our data in the Azure open AI chat playground, the chatbot only accesses the data and forgets all about the data it had with chat GPT 4.
• We, not only want to add new files, but we also want to do prompts like training so that it can understand what tech data to access and when to access it (something like a custom GPT)
• Lastly, the web app that we have deployed for the current Azure open AI service using the "deploy" button on the chat playground seems to not show up when we reaccess the playground after let's say a day or two. it asks us to deploy it as a new web app again even though we had deployed it before.

What we want to do:

• We want to train the model on custom files and data that sit on top of the existing chat GPT framework using Azure Open AI
• We want to know what combination of services to use and guidance or references to navigate our way on creating the chatbot

Answer 1

Creating a domain-specific chatbot using Azure's Open AI service is an exciting project! However, as you've discovered, the process can be challenging. Here are some potential solutions to your issues: Accessing Data and Pretrained Knowledge: Using the Retrieve and Rank (RAG) methodology, your chatbot should indeed use your provided data in combination with its pre-existing knowledge. However, it's important to note that when the conversation involves a search query, the model will primarily use the fetched documents to respond, rather than relying on its own internal knowledge. Training on Custom Files: As of now (last update in 2023), fine-tuning or direct training of GPT models on custom data is not available in Azure Cognitive Service. You can only add data sources from which the model retrieves information to answer specific queries in the conversation. Web App Deployment Disappearing: It seems unusual for the deployed Web App to disappear and require redeployment. It might be a temporary issue or a glitch on the platform. If this continues, please reach out to Azure support for direct assistance.

Given these constraints, here's what you can do to create your tech news assistant: Retrieval Over Ingestion Model: Since GPT can't be trained on custom data in Azure, consider adopting a "retrieval over ingestion" approach where you regularly update your data source (in your case, Blob Storage) with the latest tech news articles. The updated data source can then be indexed by Azure Cognitive Search, allowing the chatbot to retrieve up-to-date information from those articles. Improve Retrieval with Tags & Metadata: You can improve the retrieval effectiveness by introducing tags and metadata in your data. For example, each tech news article could have metadata about the topics covered, published date, author, etc. This would help in fetching more accurate documents relevant to user queries. Use QnA Maker for Frequently Asked Questions: For common queries or standard responses, consider using another service like Azure QnA Maker. You can feed it pairs of questions and answers related to your domain. During a conversation, if a user asks something similar, the bot will provide the corresponding answer.

For your next steps: Consider adopting a "retrieval over ingestion" approach and updating your data source regularly. Investigate improving data retrieval with metadata and tags. Explore using Azure's QnA Maker for handling frequently asked questions.

Please let me know if this helps or if you need further clarification or guidance!