Share via

Need help understanding the new format for question answering fine tuning data requirements

Frank Iacono 0 Reputation points
2024-06-05T21:01:39.98+00:00

See the new data requirements below for question / answer fine tuning. This is a radical departure from the way things have been done as far as I can tell. Every single training video that I can find uses this format or something similar.

"messages": [{"role": "system", "content": "You are a customer service chat bot."}, {"role": "user", "content": "Question?"}, {"role": "assistant", "content": "Answer."}]}

Why the big change? Why would Azure make such a radical departure without providing accessible and easy to understand documentation? Doesn't this violate their own principles and responsible AI? Can someone make it make sense? What is this Context field? What does it need to have in it? Does Context include what used to be in the content field? In general, how do we think about the relationship between the old data fields and the new data fields? I have lost five hours of my life trying to find explanations on videos and in the Azure documentation. Nothing. Can someone please help?

User's image

Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,723 questions
Azure Startups
Azure Startups
Azure: A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.Startups: Companies that are in their initial stages of business and typically developing a business model and seeking financing.
240 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Paroksh 0 Reputation points
    2024-06-06T09:23:27.68+00:00

    The new Azure Q&A format uses a structured messages array. Here's a quick guide:

    NEW FORMAT-

    {

    "messages": [

    {"role": "system", "content": "You are a customer service chat bot."},

{"role": "user", "content": "Question?"},

{"role": "assistant", "content": "Answer."}

    ]

    }

    OLD FORMAT

    {

    "context": "You are a customer service chat bot.",

    "content": "Question?",

    "response": "Answer."

    }

    Why the Change?

    • Clarity: Clearly separates roles (system, user, assistant).
    • Flexibility: Supports more complex interactions.

    Context Field

    Purpose: system role sets the scenario (e.g., "You are a customer service chat bot.").

    Content: Instructions or setup for the assistant.

    • Why the Change? Clarity: Clearly separates roles (system, user, assistant). Flexibility: Supports more complex interactions. Context Field Purpose: system role sets the scenario (e.g., "You are a customer service chat bot."). Content: Instructions or setup for the assistant
    0 comments