Events
Take the Microsoft Learn Challenge
Nov 19, 11 PM - Jan 10, 11 PM
Ignite Edition - Build skills in Microsoft Azure and earn a digital badge by January 10!
Register nowThis browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Stored completions allow you to capture the conversation history from chat completions sessions to use as datasets for evaluations and fine-tuning.
2024-10-01-preview
gpt-4o-2024-08-06
To enable stored completions for your Azure OpenAI deployment set the store
parameter to True
. Use the metadata
parameter to enrich your stored completion dataset with additional information.
import os
from openai import AzureOpenAI
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
token_provider = get_bearer_token_provider(
DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"
)
client = AzureOpenAI(
azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"),
azure_ad_token_provider=token_provider,
api_version="2024-10-01-preview"
)
completion = client.chat.completions.create(
model="gpt-4o", # replace with model deployment name
store= True,
metadata = {
"user": "admin",
"category": "docs-test",
},
messages=[
{"role": "system", "content": "Provide a clear and concise summary of the technical content, highlighting key concepts and their relationships. Focus on the main ideas and practical implications."},
{"role": "user", "content": "Ensemble methods combine multiple machine learning models to create a more robust and accurate predictor. Common techniques include bagging (training models on random subsets of data), boosting (sequentially training models to correct previous errors), and stacking (using a meta-model to combine base model predictions). Random Forests, a popular bagging method, create multiple decision trees using random feature subsets. Gradient Boosting builds trees sequentially, with each tree focusing on correcting the errors of previous trees. These methods often achieve better performance than single models by reducing overfitting and variance while capturing different aspects of the data."}
]
)
print(completion.choices[0].message)
Once stored completions are enabled for an Azure OpenAI deployment, they'll begin to show up in the Azure AI Foundry portal in the Stored Completions pane.
Distillation allows you to turn your stored completions into a fine-tuning dataset. A common use case is to use stored completions with a larger more powerful model for a particular task and then use the stored completions to train a smaller model on high quality examples of model interactions.
Distillation requires a minimum of 10 stored completions, though it's recommended to provide hundreds to thousands of stored completions for the best results.
From the Stored Completions pane in the Azure AI Foundry portal use the Filter options to select the completions you want to train your model with.
To begin distillation, select Distill
Pick which model you would like to fine-tune with your stored completion dataset.
Confirm which version of the model you want to fine-tune:
A .jsonl
file with a randomly generated name will be created as a training dataset from your stored completions. Select the file > Next.
Note
Stored completion distillation training files cannot be accessed directly and cannot be exported externally/downloaded.
The rest of the steps correspond to the typical Azure OpenAI fine-tuning steps. To learn more, see our fine-tuning getting started guide.
The evaluation of large language models is a critical step in measuring their performance across various tasks and dimensions. This is especially important for fine-tuned models, where assessing the performance gains (or losses) from training is crucial. Thorough evaluations can help your understanding of how different versions of the model may impact your application or scenario.
Stored completions can be used as a dataset for running evaluations.
From the Stored Completions pane in the Azure AI Foundry portal use the Filter options to select the completions you want to be part of your evaluation dataset.
To configure the evaluation, select Evaluate
This launches the Evaluations pane with a prepopulated .jsonl
file with a randomly generated name that is created as an evaluation dataset from your stored completions.
Note
Stored completion evaluation data files cannot be accessed directly and cannot be exported externally/downloaded.
To learn more about evaluation see, getting started with evaluations
Stored completions access is controlled via two DataActions:
Microsoft.CognitiveServices/accounts/OpenAI/stored-completions/read
Microsoft.CognitiveServices/accounts/OpenAI/stored-completions/action
By default Cognitive Services OpenAI Contributor
has access to both these permissions:
Data can be deleted by deleting the associated Azure OpenAI resource. If you wish to only delete stored completion data you must open a case with customer support.
You can store a maximum 10 GB of data.
You'll need to open a case with customer support to disable stored completions at the subscription level.
This error occurs when you're running an older version of the OpenAI client library that predates the stored completions feature being released. Run pip install openai --upgrade
.
Events
Take the Microsoft Learn Challenge
Nov 19, 11 PM - Jan 10, 11 PM
Ignite Edition - Build skills in Microsoft Azure and earn a digital badge by January 10!
Register nowTraining
Module
Generate text and conversations with .NET and Azure OpenAI Completions - Training
Learn how to use the .NET SDK with the Azure OpenAI service to have your applications carry on conversations with users in natural language.
Certification
Microsoft Certified: Azure Data Scientist Associate - Certifications
Manage data ingestion and preparation, model training and deployment, and machine learning solution monitoring with Python, Azure Machine Learning and MLflow.