Looking for internal docs or guidelines and best practices when using own support documents tuning our models

Question

We're researching into the best way to update an AI model with our latest documentation and training. Right now we have a 3.5 Turbo 32k model which we have the ability to use the Add Data Preview. Here we' are experimenting with taking our public documentation as well as our internal markdown TSG articles from our wiki.

Uploading them to a blob and then using the feature it created an index.

However what we're trying to figure out is how to go about this properly with the end goal of submitting a detailed questions about a problem, along with any azure service configuration information. Then fine tune the model to again identify the problem and only provide relevant solutions.

For example, an azure APIM service on an internal vnet is giving an error, failure to connect to the management endpoint on port 3443.

Our goal is to form that question, go out and get the vnet configuration information such as all the NSG info, route table values, peering etc.

Then if for example port 3443 inbound allow route is missing, identify that as the problem. If the route table for example only has a default route of 0.0.0.0 next hop NVA, then the response is also to create a UDR using service tag API management.

Right now, it's hit or miss no matter how we try to update information to try and train. For example, it may catch the above but instead of correcting stating to add the udr for the APIM Service tag, it will respond with make sure the NVA allows port 3443. Then also, it can start adding in other nonrelevant info that we've added to our docs which are indexed. Such as will also even respond to check DNS even if you state there are no dns issues.

Right now our options are using the dataset option to upload prompt/confirmation jsonl files which we're experimenting with.

Then we're creating .md files and trying to understand how format this to start improving the responses. Questions like do we create a single document on vnet integration and start explaining all the scenarios? Or do we have multiple documents, one that may go into details on this type of scenario. But then separate other scenarios such as DNS failures and symptoms and resolutions.

Also, we're looking for info on how to build the questions. For example like above the thought process is take the customer problem description, massage it, but then also if relevant go out and get all the info from other tools and build a complete, detailed question providing all the relevant configuration information. Is that the way to go or is the better success trying to create shorter questions and more of a back and forth type conversation?

Then we have new products coming out which we're going to have to do something because obviously data for them are just now getting produced.

Anything related to along the lines of this other groups are doing, using for guidance and training would be greatly appreciated. Also, any internal teams channels or aliases who discover this would be appreciated.

Thank you.