Learn how to customize a model for your application

The Azure OpenAI Service lets you tailor our models to your personal datasets using a process known as fine-tuning. This customization step will let you get more out of the service by providing:

  • Higher quality results than what you can get just from prompt design
  • The ability to train on more examples than can fit into a prompt
  • Lower-latency requests

A customized model improves on the few-shot learning approach by training the model's weights on your specific prompts and structure. The customized model lets you achieve better results on a wider number of tasks without needing to provide examples in your prompt. The result is less text sent and fewer tokens processed on every API call, saving cost and improving request latency.

Note

There is a breaking change in the create fine tunes command in the latest 12-01-2022 GA API. For the latest command syntax consult the reference documentation

Prerequisites

Fine-tuning workflow

The fine-tuning workflow in Azure OpenAI Studio requires the following steps:

  1. Prepare your training and validation data
  2. Use the Create customized model wizard in Azure OpenAI Studio to train your customized model
    1. Select a base model
    2. Choose your training data
    3. Optionally, choose your validation data
    4. Optionally, choose advanced options for your fine-tune job
    5. Review your choices and train your new customized model
  3. Check the status of your customized model
  4. Deploy your customized model for use
  5. Use your customized model
  6. Optionally, analyze your customized model for performance and fit

Prepare your training and validation data

Your training data and validation data sets consist of input & output examples for how you would like the model to perform.

The training and validation data you use must be formatted as a JSON Lines (JSONL) document in which each line represents a single prompt-completion pair. The OpenAI command-line interface (CLI) includes a data preparation tool that validates, gives suggestions, and reformats your training data into a JSONL file ready for fine-tuning.

Here's an example of the training data format:

{"prompt": "<prompt text>", "completion": "<ideal generated text>"}
{"prompt": "<prompt text>", "completion": "<ideal generated text>"}
{"prompt": "<prompt text>", "completion": "<ideal generated text>"}

In addition to the JSONL format, training and validation data files must be encoded in UTF-8 and include a byte-order mark (BOM), and the file must be less than 200 MB in size. For more information about formatting your training data, see Learn how to prepare your dataset for fine-tuning.

Creating your training and validation datasets

Designing your prompts and completions for fine-tuning is different from designing your prompts for use with any of our GPT-3 base models. Prompts for completion calls often use either detailed instructions or few-shot learning techniques, and consist of multiple examples. For fine-tuning, we recommend that each training example consists of a single input prompt and its desired completion output. You don't need to give detailed instructions or multiple completion examples for the same prompt.

The more training examples you have, the better. We recommend having at least 200 training examples. In general, we've found that each doubling of the dataset size leads to a linear increase in model quality.

For more information about preparing training data for various tasks, see Learn how to prepare your dataset for fine-tuning.

OpenAI CLI data preparation tool

We recommend using OpenAI's command-line interface (CLI) to assist with many of the data preparation steps. OpenAI has developed a tool that validates, gives suggestions, and reformats your data into a JSONL file ready for fine-tuning.

To install the CLI, run the following Python command:

pip install --upgrade openai 

To analyze your training data with the data preparation tool, run the following Python command, replacing <LOCAL_FILE> with the full path and file name of the training data file to be analyzed:

openai tools fine_tunes.prepare_data -f <LOCAL_FILE>

This tool accepts files in the following data formats, if they contain a prompt and a completion column/key:

  • Comma-separated values (CSV)
  • Tab-separated values (TSV)
  • Microsoft Excel workbook (XLSX)
  • JavaScript Object Notation (JSON)
  • JSON Lines (JSONL)

The tool reformats your training data and saves output into a JSONL file ready for fine-tuning, after guiding you through the process of implementing suggested changes.

Use the Create customized model wizard

Azure OpenAI Studio provides the Create customized model wizard, so you can interactively create and train a fine-tuned model for your Azure resource.

Go to the Azure OpenAI Studio

Navigate to the Azure OpenAI Studio at https://oai.azure.com/and sign in with credentials that have access to your Azure OpenAI resource. During the sign-in workflow, select the appropriate directory, Azure subscription, and Azure OpenAI resource.

Landing page

You'll first land on our main page for the Azure OpenAI Studio. From here, you can start fine-tuning a custom model.

Select the Start fine-tuning a custom model button under Manage deployments and models section of the landing page, highlighted in the following picture, to start fine-tuning a custom model.

Note

If your resource doesn't have a model already deployed in it, a warning is displayed. You can ignore that warning for the purposes of fine-tuning a model, because you'll be fine-tuning and deploying a new customized model.

Screenshot of the landing page of the Azure OpenAI Studio with sections highlighted.

Start the wizard from the Models page

To create a customized model, select the Create customized model button under the Provided models section on the Models page, highlighted in the following picture, to start the Create customized model wizard.

Screenshot of the Models page from Azure OpenAI Studio, with sections highlighted.

Select a base model

The first step in creating a customized model is to choose a base model. The Base model pane lets you choose a base model to use for your customized model, and the choice influences both the performance and the cost of your model. You can create a customized model from one of the following available base models:

  • ada
  • babbage
  • curie
  • code-cushman-001*
  • davinci* * available by request

For more information about our base models, see Models. Select a base model from the Base model type dropdown, as shown in the following picture, and then select Next to continue.

Screenshot of the Base model pane for the Create customized model wizard.

Choose your training data

The next step is to either choose existing prepared training data or upload new prepared training data to use when customizing your model. The Training data pane, shown in the following picture, displays any existing, previously uploaded datasets and provides options by which you can upload new training data.

Screenshot of the Training data pane for the Create customized model wizard.

If your training data has already been uploaded to the service, select Choose dataset, and then select the file from the list shown in the Training data pane. Otherwise, select either Local file to upload training data from a local file, or Azure blob or other shared web locations to import training data from Azure Blob or another shared web location.

For large data files, we recommend you import from an Azure Blob store. Large files can become unstable when uploaded through multipart forms because the requests are atomic and can't be retried or resumed. For more information about Azure Blob storage, see What is Azure Blob storage?

Note

Training data files must be formatted as JSONL files, encoded in UTF-8 with a byte-order mark (BOM), and less than 200 MB in size.

To upload training data from a local file

You can upload a new training dataset to the service from a local file by using one of the following methods:

  • Drag and drop the file into the client area of the Training data pane, and then select Upload file
  • Select Browse for a file from the client area of the Training data pane, choose the file to upload from the Open dialog, and then select Upload file.

After you've selected and uploaded the training dataset, select Next to optionally choose your validation data.

Screenshot of the Training data pane for the Create customized model wizard, with local file options.

To import training data from an Azure Blob store

You can import a training dataset from Azure Blob or another shared web location by providing the name and location of the file, as shown in the following picture. Enter the name of the file in File name and the Azure Blob URL, Azure Storage shared access signature (SAS), or other link to an accessible shared web location that contains the file in File location, then select Upload file to import the training dataset to the service.

After you've selected and uploaded the training dataset, select Next to optionally choose your validation data.

Screenshot of the Training data pane for the Create customized model wizard, with Azure Blob and shared web location options.

Choose your validation data

You can now choose to optionally use validation data in the training process of your fine-tuned model. If you don't want to use validation data, you can choose Next to choose advanced options for your model. Otherwise, if you have a validation dataset, you can either choose existing prepared validation data or upload new prepared validation data to use when customizing your model. The Validation data pane, shown in the following picture, displays any existing, previously uploaded training and validation datasets and provides options by which you can upload new validation data.

Screenshot of the Validation data pane for the Create customized model wizard.

If your validation data has already been uploaded to the service, select Choose dataset, and then select the file from the list shown in the Validation data pane. Otherwise, select either Local file to upload validation data from a local file, or Azure blob or other shared web locations to import validation data from Azure Blob or another shared web location.

For large data files, we recommend you import from an Azure Blob store. Large files can become unstable when uploaded through multipart forms because the requests are atomic and can't be retried or resumed.

Note

Like training data files, validation data files must be formatted as JSONL files, encoded in UTF-8 with a byte-order mark (BOM), and less than 200 MB in size.

To upload validation data from a local file

You can upload a new validation dataset to the service from a local file by using one of the following methods:

  • Drag and drop the file into the client area of the Validation data pane, and then select Upload file
  • Select Browse for a file from the client area of the Validation data pane, choose the file to upload from the Open dialog, and then select Upload file.

After you've uploaded the validation dataset, select Next to optionally choose advanced options.

Screenshot of the Validation data pane for the Create customized model wizard, with local file options.

To import validation data from an Azure Blob store

You can import a validation dataset from Azure Blob or another shared web location by providing the name and location of the file, as shown in the following picture. Enter the name of the file in File name and the Azure Blob URL, Azure Storage shared access signature (SAS), or other link to an accessible shared web location that contains the file in File location, then select Upload file to import the validation dataset to the service.

After you've imported the validation dataset, select Next to optionally choose advanced options.

Screenshot of the Validation data pane for the Create customized model wizard, with Azure Blob and shared web location options.

Choose advanced options

You can either use default values for the hyperparameters of the fine-tune job that the wizard runs to train your fine-tuned model, or you can adjust those hyperparameters for your customization needs in the Advanced options pane, shown in the following picture.

Screenshot of the Advanced options pane for the Create customized model wizard, with default options selected.

Either select Default to use the default values for the fine-tune job, or select Advanced to display and edit the hyperparameter values, as shown in the following picture.

Screenshot of the Advanced options pane for the Create customized model wizard, with advanced options selected.

The following hyperparameters are available:

Parameter name Description
Number of epochs The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset.
Batch size The batch size to use for training. The batch size is the number of training examples used to train a single forward and backward pass.
Learning rate multiplier The learning rate multiplier to use for training. The fine-tuning learning rate is the original learning rate used for pre-training, multiplied by this value.
Prompt loss weight The weight to use for loss on the prompt tokens. This value controls how much the model tries to learn to generate the prompt (as compared to the completion, which always has a weight of 1.0.) Increasing this value can add a stabilizing effect to training when completions are short.

For more information about these hyperparameters, see the Create a Fine tune job section of the REST API documentation.

After you've chosen either default or advanced options, select Next to review your choices and train your fine-tuned model.

Review your choices and train your model

The Review and train pane of the wizard displays information about the choices you've made in the Create customized model wizard for your fine-tuned model, as shown in the following picture.

Screenshot of the Review and train pane for the Create customized model wizard.

If you're ready to train your model, select Save and close to start the fine-tune job and return to the Models page.

Check the status of your customized model

The Models page displays information about your customized model in the Customized models tab, as shown in the following picture. The tab includes information about the status and job ID of the fine-tune job for your customized model. When the job is completed, the file ID of the result file is also displayed.

Screenshot of the Models page from Azure OpenAI Studio, with a customized model displayed.

After you've started a fine-tune job, it may take some time to complete. Your job may be queued behind other jobs on our system, and training your model can take minutes or hours depending on the model and dataset size. You can check the status of the fine-tune job for your customized model in the Status column of the Customized models tab on the Models page, and you can select Refresh to update the information on that page.

You can also select the name of the model from the Model name column of the Models page to display more information about your customized model, including the status of the fine-tune job, training results, training events, and hyperparameters used in the job. You can select the Refresh button to refresh the information for your model, as shown in the following picture.

Screenshot of the model page from Azure OpenAI Studio, with a customized model displayed.

From the model page, you can also select Download training file to download the training data you used for the model, or select Download results to download the result file attached to the fine-tune job for your model and analyze your customized model for training and validation performance.

Deploy a customized model

When the fine-tune job has succeeded, you can deploy the customized model from the Models pane. You must deploy your customized model to make it available for use with completion calls.

Note

Only one deployment is permitted for a customized model. An error message is displayed if you select an already-deployed customized model.

To deploy your customized model, select the customized model to be deployed and then select Deploy model, as shown in the following picture.

Screenshot of the Models page from Azure OpenAI Studio, with the Deploy model button highlighted.

The Deploy model dialog is presented, in which you can provide a name for the deployment of your customized model. Enter a name in Deployment name and then select Create to start the deployment of your customized model.

Screenshot of the Deploy Model dialog from Azure OpenAI Studio.

You can monitor the progress of your deployment from the Deployments pane of Azure OpenAI Studio.

Use a deployed customized model

Once your customized model has been deployed, you can use it like any other deployed model. For example, you can use the Playground pane of Azure OpenAI Studio to experiment with your new deployment, as shown in the following picture. You can continue to use the same parameters with your customized model, such as temperature and frequency penalty, as you can with other deployed models.

Screenshot of the Playground page of Azure OpenAI Studio, with sections highlighted.

Note

As with all applications, we require a review process prior to going live.

Analyze your customized model

Azure OpenAI attaches a result file, named results.csv, to each fine-tune job once it's completed. You can use the result file to analyze the training and validation performance of your customized model. The file ID for the result file is listed for each customized model in the Result file Id column of the Models pane for Azure OpenAI Studio. You can use the file ID to identify and download the result file from the File Management pane of Azure OpenAI Studio.

The result file is a CSV file containing a header row and a row for each training step performed by the fine-tune job. The result file contains the following columns:

Column name Description
step The number of the training step. A training step represents a single pass, forward and backward, on a batch of training data.
elapsed_tokens The number of tokens the customized model has seen so far, including repeats.
elapsed_examples The number of examples the model has seen so far, including repeats.
Each example represents one element in that step's batch of training data. For example, if the Batch size parameter is set to 32 in the Advanced options pane, this value increments by 32 in each training step.
training_loss The loss for the training batch.
training_sequence_accuracy The percentage of completions in the training batch for which the model's predicted tokens exactly matched the true completion tokens.
For example, if the batch size is set to 3 and your data contains completions [[1, 2], [0, 5], [4, 2]], this value is set to 0.67 (2 of 3) if the model predicted [[1, 1], [0, 5], [4, 2]].
training_token_accuracy The percentage of tokens in the training batch that were correctly predicted by the model.
For example, if the batch size is set to 3 and your data contains completions [[1, 2], [0, 5], [4, 2]], this value is set to 0.83 (5 of 6) if the model predicted [[1, 1], [0, 5], [4, 2]].
validation_loss The loss for the validation batch.
validation_sequence_accuracy The percentage of completions in the validation batch for which the model's predicted tokens exactly matched the true completion tokens.
For example, if the batch size is set to 3 and your data contains completions [[1, 2], [0, 5], [4, 2]], this value is set to 0.67 (2 of 3) if the model predicted [[1, 1], [0, 5], [4, 2]].
validation_token_accuracy The percentage of tokens in the validation batch that were correctly predicted by the model.
For example, if the batch size is set to 3 and your data contains completions [[1, 2], [0, 5], [4, 2]], this value is set to 0.83 (5 of 6) if the model predicted [[1, 1], [0, 5], [4, 2]].

Clean up your deployments, customized models, and training files

When you're done with your customized model, you can delete the deployment and model. You can also delete the training and validation files you uploaded to the service, if needed.

Delete your model deployment

You can delete the deployment for your customized model from the Deployments page for Azure OpenAI Studio. Select the deployment to delete, and then select Delete to delete the deployment.

Delete your customized model

You can delete a customized model from the Models page for Azure OpenAI Studio. Select the customized model to delete from the Customized models tab, and then select Delete to delete the customized model.

Note

You cannot delete a customized model if it has an existing deployment. You must first delete your model deployment before you can delete your customized model.

Delete your training files

You can optionally delete training and validation files you've uploaded for training, and result files generated during training, from the File Management page for Azure OpenAI Studio. Select the file to delete, and then select Delete to delete the file.

Next steps

Library source code | Package (PyPi) |

Prerequisites

  • An Azure subscription - Create one for free

  • Access granted to the Azure OpenAI service in the desired Azure subscription

    Currently, access to this service is granted only by application. You can apply for access to the Azure OpenAI service by completing the form at https://aka.ms/oai/access. Open an issue on this repo to contact us if you have an issue.

  • An Azure OpenAI resource

    For more information about creating a resource, see Create a resource and deploy a model using Azure OpenAI.

  • The following Python libraries: os, json

Fine-tuning workflow

The fine-tuning workflow when using the Python SDK with Azure OpenAI requires the following steps:

  1. Prepare your training and validation data
  2. Select a base model
  3. Upload your training data
  4. Train your new customized model
  5. Check the status of your customized model
  6. Deploy your customized model for use
  7. Use your customized model
  8. Optionally, analyze your customized model for performance and fit

Prepare your training and validation data

Your training data and validation data sets consist of input & output examples for how you would like the model to perform.

The training and validation data you use must be formatted as a JSON Lines (JSONL) document in which each line represents a single prompt-completion pair. The OpenAI command-line interface (CLI) includes a data preparation tool that validates, gives suggestions, and reformats your training data into a JSONL file ready for fine-tuning.

Here's an example of the training data format:

{"prompt": "<prompt text>", "completion": "<ideal generated text>"}
{"prompt": "<prompt text>", "completion": "<ideal generated text>"}
{"prompt": "<prompt text>", "completion": "<ideal generated text>"}

In addition to the JSONL format, training and validation data files must be encoded in UTF-8 and include a byte-order mark (BOM), and the file must be less than 200 MB in size. For more information about formatting your training data, see Learn how to prepare your dataset for fine-tuning.

Creating your training and validation datasets

Designing your prompts and completions for fine-tuning is different from designing your prompts for use with any of our GPT-3 base models. Prompts for completion calls often use either detailed instructions or few-shot learning techniques, and consist of multiple examples. For fine-tuning, we recommend that each training example consists of a single input prompt and its desired completion output. You don't need to give detailed instructions or multiple completion examples for the same prompt.

The more training examples you have, the better. We recommend having at least 200 training examples. In general, we've found that each doubling of the dataset size leads to a linear increase in model quality.

For more information about preparing training data for various tasks, see Learn how to prepare your dataset for fine-tuning.

OpenAI CLI data preparation tool

We recommend using OpenAI's command-line interface (CLI) to assist with many of the data preparation steps. OpenAI has developed a tool that validates, gives suggestions, and reformats your data into a JSONL file ready for fine-tuning.

To install the CLI, run the following Python command:

pip install --upgrade openai 

To analyze your training data with the data preparation tool, run the following Python command, replacing <LOCAL_FILE> with the full path and file name of the training data file to be analyzed:

openai tools fine_tunes.prepare_data -f <LOCAL_FILE>

This tool accepts files in the following data formats, if they contain a prompt and a completion column/key:

  • Comma-separated values (CSV)
  • Tab-separated values (TSV)
  • Microsoft Excel workbook (XLSX)
  • JavaScript Object Notation (JSON)
  • JSON Lines (JSONL)

The tool reformats your training data and saves output into a JSONL file ready for fine-tuning, after guiding you through the process of implementing suggested changes.

Select a base model

The first step in creating a customized model is to choose a base model. The choice influences both the performance and the cost of your model. You can create a customized model from one of the following available base models:

  • ada
  • babbage
  • curie
  • code-cushman-001*
  • davinci* * available by request

You can use the Models API to identify which models are fine-tunable. For more information about our base models, see Models.

Upload your training data

The next step is to either choose existing prepared training data or upload new prepared training data to use when customizing your model. Once you've prepared your training data, you can upload your files to the service. We offer two ways to upload training data:

For large data files, we recommend you import from an Azure Blob store. Large files can become unstable when uploaded through multipart forms because the requests are atomic and can't be retried or resumed. For more information about Azure Blob storage, see What is Azure Blob storage?

Note

Training data files must be formatted as JSONL files, encoded in UTF-8 with a byte-order mark (BOM), and less than 200 MB in size.

The following Python example locally creates sample training and validation dataset files, then uploads the local files using the Python SDK and retrieves the returned file IDs. Make sure to save the IDs returned by the example, because you'll need them for the fine-tuning training job creation.

Important

Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like Azure Key Vault. See the Cognitive Services security article for more information.

import openai
from openai import cli
import time
import shutil
import json

# Remember to remove your key from your code when you're done.
openai.api_key = "COPY_YOUR_OPENAI_KEY_HERE"
# Your resource endpoint should look like the following:
# https://YOUR_RESOURCE_NAME.openai.azure.com/
openai.api_base =  "COPY_YOUR_OPENAI_ENDPOINT_HERE" 
openai.api_type = 'azure'
# The API version may change in the future.
openai.api_version = '2022-06-01-preview'

training_file_name = 'training.jsonl'
validation_file_name = 'validation.jsonl'

sample_data = [{"prompt": "When I go to the store, I want an", "completion": "apple"},
    {"prompt": "When I go to work, I want a", "completion": "coffee"},
    {"prompt": "When I go home, I want a", "completion": "soda"}]

# Generate the training dataset file.
print(f'Generating the training file: {training_file_name}')
with open(training_file_name, 'w') as training_file:
    for entry in sample_data:
        json.dump(entry, training_file)
        training_file.write('\n')

# Copy the validation dataset file from the training dataset file.
# Typically, your training data and validation data should be mutually exclusive.
# For the purposes of this example, we're using the same data.
print(f'Copying the training file to the validation file')
shutil.copy(training_file_name, validation_file_name)

def check_status(training_id, validation_id):
    train_status = openai.File.retrieve(training_id)["status"]
    valid_status = openai.File.retrieve(validation_id)["status"]
    print(f'Status (training_file | validation_file): {train_status} | {valid_status}')
    return (train_status, valid_status)

# Upload the training and validation dataset files to Azure OpenAI.
training_id = cli.FineTune._get_or_upload(training_file_name, True)
validation_id = cli.FineTune._get_or_upload(validation_file_name, True)

# Check on the upload status of the training and validation dataset files.
(train_status, valid_status) = check_status(training_id, validation_id)

# Poll and display the upload status once a second until both files have either
# succeeded or failed to upload.
while train_status not in ["succeeded", "failed"] or valid_status not in ["succeeded", "failed"]:
    time.sleep(1)
    (train_status, valid_status) = check_status(training_id, validation_id)

Create a customized model

After you've uploaded your training and validation files, you're ready to start the fine-tune job. The following Python code shows an example of how to create a new fine-tune job with the Python SDK:

# This example defines a fine-tune job that creates a customized model based on curie, 
# with just a single pass through the training data. The job also provides classification-
# specific metrics, using our validation data, at the end of that epoch.
create_args = {
    "training_file": training_id,
    "validation_file": validation_id,
    "model": "curie",
    "hyperparams": {
    "n_epochs": 1
    },
    "compute_classification_metrics": True,
    "classification_n_classes": 3
}
# Create the fine-tune job and retrieve the job ID
# and status from the response.
resp = openai.FineTune.create(**create_args)
job_id = resp["id"]
status = resp["status"]

# You can use the job ID to monitor the status of the fine-tune job.
# The fine-tune job may take some time to start and complete.
print(f'Fine-tuning model with job ID: {job_id}.')

You can either use default values for the hyperparameters of the fine-tune job, or you can adjust those hyperparameters for your customization needs. For the previous Python example, we've set the n_epochs hyperparameter to 1, indicating that we want just one full cycle through the training data. For more information about these hyperparameters, see the Create a Fine tune job section of the REST API documentation.

Check the status of your customized model

After you've started a fine-tune job, it may take some time to complete. Your job may be queued behind other jobs on our system, and training your model can take minutes or hours depending on the model and dataset size. The following Python example checks the status of your fine-tune job by retrieving information about your job using the job ID returned from the previous example:

# Get the status of our fine-tune job.
status = openai.FineTune.retrieve(id=job_id)["status"]

# If the job isn't yet done, poll it every 2 seconds.
if status not in ["succeeded", "failed"]:
    print(f'Job not in terminal status: {status}. Waiting.')
    while status not in ["succeeded", "failed"]:
        time.sleep(2)
        status = openai.FineTune.retrieve(id=job_id)["status"]
        print(f'Status: {status}')
else:
    print(f'Fine-tune job {job_id} finished with status: {status}')

# Check if there are other fine-tune jobs in the subscription. 
# Your fine-tune job may be queued, so this is helpful information to have
# if your fine-tune job hasn't yet started.
print('Checking other fine-tune jobs in the subscription.')
result = openai.FineTune.list()
print(f'Found {len(result)} fine-tune jobs.')

Deploy a customized model

When the fine-tune job has succeeded, the value of fine_tuned_model in the response body of the FineTune.retrieve() method is set to the name of your customized model. Your model is now also available for discovery from the list Models API. However, you can't issue completion calls to your customized model until your customized model is deployed. You must deploy your customized model to make it available for use with completion calls.

Note

As with all applications, we require a review process prior to going live.

You can use either Azure OpenAI or the Azure Command-Line Interface (CLI) to deploy your customized model.

Note

Only one deployment is permitted for a customized model. An error occurs if you select an already-deployed customized model.

Deploy a model with Azure OpenAI

The following Python example shows how to use the deployment API included with Azure OpenAI to create a model deployment for your customized model. The deployment API generates a name for the deployment of your customized model.

# Retrieve the name of the customized model from the fine-tune job.
result = openai.FineTune.retrieve(id=job_id)
if result["status"] == 'succeeded':
    model = result["fine_tuned_model"]

# Create the deployment for the customized model, using the standard scale type without specifying a scale
# capacity.
print(f'Creating a new deployment with model: {model}')
result = openai.Deployment.create(model=model, scale_settings={"scale_type":"standard", "capacity": None})
# Retrieve the deployment job ID from the results.
deployment_id = result["id"]

Deploy a model with Azure CLI

The following Azure CLI command example shows how to use the Azure CLI to deploy your customized model. With the Azure CLI, you must specify a name for the deployment of your customized model. For more information about using the Azure CLI to deploy customized models, see az cognitiveservices account deployment in the Azure Command-Line Interface (CLI) documentation.

To run this Azure CLI command in a console window, you must replace the following placeholders with the corresponding values for your customized model:

Placeholder Value
YOUR_AZURE_SUBSCRIPTION The name or ID of your Azure subscription.
YOUR_RESOURCE_GROUP The name of your Azure resource group.
YOUR_RESOURCE_NAME The name of your Azure OpenAI resource.
YOUR_DEPLOYMENT_NAME The name you want to use for your model deployment.
YOUR_FINE_TUNED_MODEL_ID The name of your customized model.
az cognitiveservices account deployment create 
    --subscription YOUR_AZURE_SUBSCRIPTION
    -g YOUR_RESOURCE_GROUP
    -n YOUR_RESOURCE_NAME 
    --deployment-name YOUR_DEPLOYMENT_NAME
    --model-name YOUR_FINE_TUNED_MODEL_ID 
    --model-version "1" 
    --model-format OpenAI 
    --scale-settings-scale-type "Standard" 

Use a deployed customized model

Once your customized model has been deployed, you can use it like any other deployed model. For example, you can send a completion call to your deployed model, as shown in the following Python example. You can continue to use the same parameters with your customized model, such as temperature and frequency penalty, as you can with other deployed models.

print('Sending a test completion job')
start_phrase = 'When I go to the store, I want a'
response = openai.Completion.create(engine=deployment_id, prompt=start_phrase, max_tokens=4)
text = response['choices'][0]['text'].replace('\n', '').replace(' .', '.').strip()
print(f'"{start_phrase} {text}"')

Analyze your customized model

Azure OpenAI attaches a result file, named results.csv, to each fine-tune job once it's completed. You can use the result file to analyze the training and validation performance of your customized model. The file ID for the result file is listed for each customized model, and you can use the Python SDK to retrieve the file ID and download the result file for analysis.

The following Python example retrieves the file ID of the first result file attached to the fine-tune job for your customized model, and then uses the Python SDK to download the file to your working directory for analysis.

# Retrieve the file ID of the first result file from the fine-tune job for
# the customized model.
result = openai.FineTune.retrieve(id=job_id)
if result["status"] == 'succeeded':
    result_file_id = result.result_files[0].id
    result_file_name = result.result_files[0].filename

# Download the result file.
print(f'Downloading result file: {result_file_id}')
# Write the byte array returned by the File.download() method to 
# a local file in the working directory.
with open(result_file_name, "wb") as file:
    result = openai.File.download(id=result_file_id)
    file.write(result)

The result file is a CSV file containing a header row and a row for each training step performed by the fine-tune job. The result file contains the following columns:

Column name Description
step The number of the training step. A training step represents a single pass, forward and backward, on a batch of training data.
elapsed_tokens The number of tokens the customized model has seen so far, including repeats.
elapsed_examples The number of examples the model has seen so far, including repeats.
Each example represents one element in that step's batch of training data. For example, if the Batch size parameter is set to 32 in the Advanced options pane, this value increments by 32 in each training step.
training_loss The loss for the training batch.
training_sequence_accuracy The percentage of completions in the training batch for which the model's predicted tokens exactly matched the true completion tokens.
For example, if the batch size is set to 3 and your data contains completions [[1, 2], [0, 5], [4, 2]], this value is set to 0.67 (2 of 3) if the model predicted [[1, 1], [0, 5], [4, 2]].
training_token_accuracy The percentage of tokens in the training batch that were correctly predicted by the model.
For example, if the batch size is set to 3 and your data contains completions [[1, 2], [0, 5], [4, 2]], this value is set to 0.83 (5 of 6) if the model predicted [[1, 1], [0, 5], [4, 2]].
validation_loss The loss for the validation batch.
validation_sequence_accuracy The percentage of completions in the validation batch for which the model's predicted tokens exactly matched the true completion tokens.
For example, if the batch size is set to 3 and your data contains completions [[1, 2], [0, 5], [4, 2]], this value is set to 0.67 (2 of 3) if the model predicted [[1, 1], [0, 5], [4, 2]].
validation_token_accuracy The percentage of tokens in the validation batch that were correctly predicted by the model.
For example, if the batch size is set to 3 and your data contains completions [[1, 2], [0, 5], [4, 2]], this value is set to 0.83 (5 of 6) if the model predicted [[1, 1], [0, 5], [4, 2]].

Clean up your deployments, customized models, and training files

When you're done with your customized model, you can delete the deployment and model. You can also delete the training and validation files you uploaded to the service, if needed.

Delete your model deployment

You can use various methods to delete the deployment for your customized model:

The following Python example uses the Python SDK to delete the deployment for your customized model.

# Delete the deployment for the customized model
print(f'Deleting deployment ID: {deployment_id}')
result = openai.Deployment.delete(sid=deployment_id)

Delete your customized model

Similarly, you can use various methods to delete your customized model:

Note

You cannot delete a customized model if it has an existing deployment. You must first delete your model deployment before you can delete your customized model.

The following Python example uses the Python SDK to delete the deployment for your customized model.

# Delete the customized model
print(f'Deleting customized model ID: {job_id}')
result = openai.FineTune.delete(sid=job_id)

Delete your training files

You can optionally delete training and validation files you've uploaded for training, and result files generated during training, from your Azure OpenAI subscription. You can use the following methods to delete your training, validation, and result files:

The following Python example uses the Python SDK to delete the training, validation, and result files for your customized model.

print('Checking for existing uploaded files.')
results = []

# Get the complete list of uploaded files in our subscription.
files = openai.File.list().data
print(f'Found {len(files)} total uploaded files in the subscription.')

# Enumerate all uploaded files, extracting the file IDs for the
# files with file names that match your training dataset file and
# validation dataset file names.
for item in files:
    if item["filename"] in [training_file_name, validation_file_name, result_file_name]:
        results.append(item["id"])
print(f'Found {len(results)} already uploaded files that match our files')

# Enumerate the file IDs for our files and delete each file.
print(f'Deleting already uploaded files.')
for id in results:
    openai.File.delete(sid = id)

Next steps

Prerequisites

  • An Azure subscription - Create one for free

  • Access granted to the Azure OpenAI service in the desired Azure subscription

    Currently, access to this service is granted only by application. You can apply for access to the Azure OpenAI service by completing the form at https://aka.ms/oai/access. Open an issue on this repo to contact us if you have an issue.

  • An Azure OpenAI resource

    For more information about creating a resource, see Create a resource and deploy a model using Azure OpenAI.

  • The following Python libraries: os, json, requests

Fine-tuning workflow

The fine-tuning workflow when using the Python SDK with Azure OpenAI requires the following steps:

  1. Prepare your training and validation data
  2. Select a base model
  3. Upload your training data
  4. Train your new customized model
  5. Check the status of your customized model
  6. Deploy your customized model for use
  7. Use your customized model
  8. Optionally, analyze your customized model for performance and fit

Prepare your training and validation data

Your training data and validation data sets consist of input & output examples for how you would like the model to perform.

The training and validation data you use must be formatted as a JSON Lines (JSONL) document in which each line represents a single prompt-completion pair. The OpenAI command-line interface (CLI) includes a data preparation tool that validates, gives suggestions, and reformats your training data into a JSONL file ready for fine-tuning.

Here's an example of the training data format:

{"prompt": "<prompt text>", "completion": "<ideal generated text>"}
{"prompt": "<prompt text>", "completion": "<ideal generated text>"}
{"prompt": "<prompt text>", "completion": "<ideal generated text>"}

In addition to the JSONL format, training and validation data files must be encoded in UTF-8 and include a byte-order mark (BOM), and the file must be less than 200 MB in size. For more information about formatting your training data, see Learn how to prepare your dataset for fine-tuning.

Creating your training and validation datasets

Designing your prompts and completions for fine-tuning is different from designing your prompts for use with any of our GPT-3 base models. Prompts for completion calls often use either detailed instructions or few-shot learning techniques, and consist of multiple examples. For fine-tuning, we recommend that each training example consists of a single input prompt and its desired completion output. You don't need to give detailed instructions or multiple completion examples for the same prompt.

The more training examples you have, the better. We recommend having at least 200 training examples. In general, we've found that each doubling of the dataset size leads to a linear increase in model quality.

For more information about preparing training data for various tasks, see Learn how to prepare your dataset for fine-tuning.

OpenAI CLI data preparation tool

We recommend using OpenAI's command-line interface (CLI) to assist with many of the data preparation steps. OpenAI has developed a tool that validates, gives suggestions, and reformats your data into a JSONL file ready for fine-tuning.

To install the CLI, run the following Python command:

pip install --upgrade openai 

To analyze your training data with the data preparation tool, run the following Python command, replacing <LOCAL_FILE> with the full path and file name of the training data file to be analyzed:

openai tools fine_tunes.prepare_data -f <LOCAL_FILE>

This tool accepts files in the following data formats, if they contain a prompt and a completion column/key:

  • Comma-separated values (CSV)
  • Tab-separated values (TSV)
  • Microsoft Excel workbook (XLSX)
  • JavaScript Object Notation (JSON)
  • JSON Lines (JSONL)

The tool reformats your training data and saves output into a JSONL file ready for fine-tuning, after guiding you through the process of implementing suggested changes.

Select a base model

The first step in creating a customized model is to choose a base model. The choice influences both the performance and the cost of your model. You can create a customized model from one of the following available base models:

  • ada
  • babbage
  • curie
  • code-cushman-001*
  • davinci* * available by request

You can use the Models API to identify which models are fine-tunable. For more information about our base models, see Models.

Upload your training data

The next step is to either choose existing prepared training data or upload new prepared training data to use when customizing your model. Once you've prepared your training data, you can upload your files to the service. We offer two ways to upload training data:

For large data files, we recommend you import from an Azure Blob store. Large files can become unstable when uploaded through multipart forms because the requests are atomic and can't be retried or resumed. For more information about Azure Blob storage, see What is Azure Blob storage?

Note

Training data files must be formatted as JSONL files, encoded in UTF-8 with a byte-order mark (BOM), and less than 200 MB in size.

The following Python example locally creates sample training and validation dataset files, then invokes the REST API to upload the local files and retrieve the returned file IDs. Make sure to save the IDs returned by the example, because you'll need them for the fine-tuning training job creation.

Important

Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like Azure Key Vault. See the Cognitive Services security article for more information.

import os
import requests
import json
import time
import shutil

# Remember to remove your key from your code when you're done.
api_key = "COPY_YOUR_OPENAI_KEY_HERE"
# Your resource endpoint should look like the following:
# https://YOUR_RESOURCE_NAME.openai.azure.com/
api_base =  "COPY_YOUR_OPENAI_ENDPOINT_HERE"
api_type = 'azure'
# The API version may change in the future.
api_version = '2022-06-01-preview'

training_file_name = 'training.jsonl'
validation_file_name = 'validation.jsonl'

sample_data = [{"prompt": "When I go to the store, I want an", "completion": "apple"},
    {"prompt": "When I go to work, I want a", "completion": "coffee"},
    {"prompt": "When I go home, I want a", "completion": "soda"}]

# Generate the training dataset file.
print(f'Generating the training file: {training_file_name}')
with open(training_file_name, 'w') as training_file:
    for entry in sample_data:
        json.dump(entry, training_file)
        training_file.write('\n')

# Copy the validation dataset file from the training dataset file.
# Typically, your training data and validation data should be mutually exclusive.
# For the purposes of this example, we're using the same data.
print(f'Copying the training file to the validation file')
shutil.copy(training_file_name, validation_file_name)

# Upload the training & validation dataset files to Azure OpenAI with the REST API.
upload_params = {'api-version': api_version}
upload_headers = {'api-key': api_key}
upload_data = {'purpose': 'fine-tune'}

# Upload the training file
r = requests.post(api_base + 'openai/files', 
      params=upload_params, headers=upload_headers, data=upload_data,
      files={'file': (
        training_file_name, 
        open(training_file_name, 'rb'),
        'application/json')
      }
    )
training_id=(r.json())["id"]

# Upload the validation file
r = requests.post(api_base + "openai/files", 
      params=upload_params, headers=upload_headers, data=upload_data,
      files={'file': (
        validation_file_name, 
        open(validation_file_name, 'rb'),
        'application/json')
      }
    )
validation_id=(r.json())["id"]

Create a customized model

After you've uploaded your training and validation files, you're ready to start the fine-tune job. The following Python code shows an example of how to create a new fine-tune job with the Python SDK:

# This example defines a fine-tune job that creates a customized model based on curie, 
# with just a single pass through the training data. The job also provides classification-
# specific metrics, using our validation data, at the end of that epoch.
# Note that the form data for this REST API method is a string representation of JSON.
fine_tune_params = {'api-version': api_version}
fine_tune_headers = {'api-key': api_key}
fine_tune_data = "{ \"model\": \"curie\", " + \
  "\"training_file\": \"" + training_id + "\", " + \
  "\"validation_file\": \"" + validation_id + "\", " + \
  "\"hyperparams\": " + \
  "{ \"batch_size\": 1, \"learning_rate_multiplier\": 0.1, \"n_epochs\": 4 } }"

# Start the fine-tune job using the REST API
r = requests.post(api_base + 'openai/fine-tunes', 
      params=fine_tune_params, headers=fine_tune_headers, data=fine_tune_data)

# Retrieve the job ID and job status from the response
job_id = (r.json())["id"]
status = (r.json())["status"]

print(f'Fine-tuning model with job ID: {job_id}.')

You can either use default values for the hyperparameters of the fine-tune job, or you can adjust those hyperparameters for your customization needs. For the previous Python example, we've set the n_epochs hyperparameter to 1, indicating that we want just one full cycle through the training data. For more information about these hyperparameters, see the Create a Fine tune job section of the REST API documentation.

Check the status of your customized model

After you've started a fine-tune job, it may take some time to complete. Your job may be queued behind other jobs on our system, and training your model can take minutes or hours depending on the model and dataset size. The following Python example uses the REST API to check the status of your fine-tune job. The example retrieves information about your job using the job ID returned from the previous example:

# Get the status of our fine-tune job.
r = requests.get(api_base + 'openai/fine-tunes/' + job_id, 
      params=fine_tune_params, headers=fine_tune_headers)

# If the job isn't yet done, poll it every 2 seconds.
status = (r.json())["status"]
if status not in ["succeeded", "failed"]:
    print(f'Job not in terminal status: {status}. Waiting.')
    while status not in ["succeeded", "failed"]:
        time.sleep(2)
        r = requests.get(api_base + 'openai/fine-tunes/' + job_id, 
              params=fine_tune_params, headers=fine_tune_headers)
        status = (r.json())["status"]
        print(f'Status: {status}')
else:
    print(f'Fine-tune job {job_id} finished with status: {status}')

# List all fine-tune jobs available in the subscription
print('Checking other fine-tune jobs in the subscription.')
r = requests.get(api_base + 'openai/fine-tunes', 
      params=fine_tune_params, headers=fine_tune_headers)
print(f'Found {len((r.json())["data"])} fine-tune jobs.')

Deploy a customized model

When the fine-tune job has succeeded, the value of fine_tuned_model in the response body of the FineTune.retrieve() method is set to the name of your customized model. Your model is now also available for discovery from the list Models API. However, you can't issue completion calls to your customized model until your customized model is deployed. You must deploy your customized model to make it available for use with completion calls.

Note

As with all applications, we require a review process prior to going live.

You can use either the REST API or the Azure Command-Line Interface (CLI) to deploy your customized model.

Note

Only one deployment is permitted for a customized model. An error occurs if you select an already-deployed customized model.

Deploy a model with the REST API

The following Python example shows how to use the REST API to create a model deployment for your customized model. The REST API generates a name for the deployment of your customized model.

# Retrieve the name of the customized model from the fine-tune job.
r = requests.get(api_base + 'openai/fine-tunes/' + job_id, 
      params=fine_tune_params, headers=fine_tune_headers)
if (r.json())["status"] == 'succeeded':
    model = (r.json())["fine_tuned_model"]

# This example creates the deployment for the customized model, using the standard
# scale type without specifying a scale capacity.
# Note that the form data for this REST API method is a string representation of JSON.
deploy_params = {'api-version': api_version}
deploy_headers = {'api-key': api_key}
deploy_data = "{ \"model\": \"" + model + "\", " + \
  "\"scale_settings\": " + \
  "{ \"scale_type\": \"standard\", \"capacity\": \"None\" } }"

print(f'Creating a new deployment with model: {model}')
r = requests.post(api_base + 'openai/deployments', 
      params=deploy_params, headers=deploy_headers, data=deploy_data
    )

# Retrieve the deployment job ID from the results.
deployment_id = (r.json())["id"]

Deploy a model with Azure CLI

The following Azure CLI command example shows how to use the Azure CLI to deploy your customized model. With the Azure CLI, you must specify a name for the deployment of your customized model. For more information about using the Azure CLI to deploy customized models, see az cognitiveservices account deployment in the Azure Command-Line Interface (CLI) documentation.

To run this Azure CLI command in a console window, you must replace the following placeholders with the corresponding values for your customized model:

Placeholder Value
YOUR_AZURE_SUBSCRIPTION The name or ID of your Azure subscription.
YOUR_RESOURCE_GROUP The name of your Azure resource group.
YOUR_RESOURCE_NAME The name of your Azure OpenAI resource.
YOUR_DEPLOYMENT_NAME The name you want to use for your model deployment.
YOUR_FINE_TUNED_MODEL_ID The name of your customized model.
az cognitiveservices account deployment create 
    --subscription YOUR_AZURE_SUBSCRIPTION
    -g YOUR_RESOURCE_GROUP
    -n YOUR_RESOURCE_NAME 
    --deployment-name YOUR_DEPLOYMENT_NAME
    --model-name YOUR_FINE_TUNED_MODEL_ID 
    --model-version "1" 
    --model-format OpenAI 
    --scale-settings-scale-type "Standard" 

Use a deployed customized model

Once your customized model has been deployed, you can use it like any other deployed model. For example, you can send a completion call to your deployed model, as shown in the following Python example. You can continue to use the same parameters with your customized model, such as temperature and frequency penalty, as you can with other deployed models.

# Send a completion call to the deployed model using the REST API.
# Note that the form data for this REST API method is a string representation of JSON.
start_phrase = 'When I go to the store, I want a'

completion_params = {'api-version': api_version}
completion_headers = {'api-key': api_key}
completion_data = "{ \"prompt\": \"" + start_phrase + "\", " + \
  "\"max_tokens\": 4 }"

print('Sending a test completion job')
r = requests.post(api_base + 'openai/deployments/' + deployment_id, 
      params=completion_params, headers=completion_headers, data=completion_data
    )

text = (r.json())['choices'][0]['text'].replace('\n', '').replace(' .', '.').strip()
print(f'"{start_phrase} {text}"')

Analyze your customized model

Azure OpenAI attaches a result file, named results.csv, to each fine-tune job once it's completed. You can use the result file to analyze the training and validation performance of your customized model. The file ID for the result file is listed for each customized model, and you can use the REST API to retrieve the file ID and download the result file for analysis.

The following Python example uses the REST API to retrieve the file ID of the first result file attached to the fine-tune job for your customized model, and then downloads the file to your working directory for analysis.

# Retrieve the file ID of the first result file from the fine-tune job for
# the customized model.
r = requests.get(api_base + 'openai/fine-tunes/' + job_id, 
      params=fine_tune_params, headers=fine_tune_headers)

if (r.json())["status"] == 'succeeded':
    result_file_id = (r.json())["result_files"][0]["id"]
    result_file_name = (r.json())["result_files"][0]["filename"]

    # Download the result file using the REST API
    result_file_params = {'api-version': api_version}
    result_file_headers = {'api-key': api_key}

    print(f'Downloading result file: {result_file_id}')
    # Write the file contents returned by the REST API method to 
    # a local file in the working directory.
    with open(result_file_name, "w") as file:
        r = requests.get(api_base + 'openai/files/' + result_file_id + "/content", 
              params=result_file_params, headers=result_file_headers)
        file.write(r.text)

The result file is a CSV file containing a header row and a row for each training step performed by the fine-tune job. The result file contains the following columns:

Column name Description
step The number of the training step. A training step represents a single pass, forward and backward, on a batch of training data.
elapsed_tokens The number of tokens the customized model has seen so far, including repeats.
elapsed_examples The number of examples the model has seen so far, including repeats.
Each example represents one element in that step's batch of training data. For example, if the Batch size parameter is set to 32 in the Advanced options pane, this value increments by 32 in each training step.
training_loss The loss for the training batch.
training_sequence_accuracy The percentage of completions in the training batch for which the model's predicted tokens exactly matched the true completion tokens.
For example, if the batch size is set to 3 and your data contains completions [[1, 2], [0, 5], [4, 2]], this value is set to 0.67 (2 of 3) if the model predicted [[1, 1], [0, 5], [4, 2]].
training_token_accuracy The percentage of tokens in the training batch that were correctly predicted by the model.
For example, if the batch size is set to 3 and your data contains completions [[1, 2], [0, 5], [4, 2]], this value is set to 0.83 (5 of 6) if the model predicted [[1, 1], [0, 5], [4, 2]].
validation_loss The loss for the validation batch.
validation_sequence_accuracy The percentage of completions in the validation batch for which the model's predicted tokens exactly matched the true completion tokens.
For example, if the batch size is set to 3 and your data contains completions [[1, 2], [0, 5], [4, 2]], this value is set to 0.67 (2 of 3) if the model predicted [[1, 1], [0, 5], [4, 2]].
validation_token_accuracy The percentage of tokens in the validation batch that were correctly predicted by the model.
For example, if the batch size is set to 3 and your data contains completions [[1, 2], [0, 5], [4, 2]], this value is set to 0.83 (5 of 6) if the model predicted [[1, 1], [0, 5], [4, 2]].

Clean up your deployments, customized models, and training files

When you're done with your customized model, you can delete the deployment and model. You can also delete the training and validation files you uploaded to the service, if needed.

Delete your model deployment

You can use various methods to delete the deployment for your customized model:

The following Python example uses the REST API to delete the deployment for your customized model.

# Delete the deployment for the customized model
print(f'Deleting model deployment ID: {deployment_id}')
r = requests.delete(api_base + 'openai/deployments/' + deployment_id,
      params=deploy_params, headers=deploy_headers)

Delete your customized model

Similarly, you can use various methods to delete your customized model:

Note

You cannot delete a customized model if it has an existing deployment. You must first delete your model deployment before you can delete your customized model.

The following Python example uses the REST API to delete the deployment for your customized model.

# Delete the customized model
print(f'Deleting customized model ID: {job_id}')
r = requests.delete(api_base + 'openai/fine-tunes/' + job_id,
      params=fine_tune_params, headers=fine_tune_headers)

Delete your training files

You can optionally delete training and validation files you've uploaded for training, and result files generated during training, from your Azure OpenAI subscription. You can use the following methods to delete your training, validation, and result files:

The following Python example uses the REST API to delete the training, validation, and result files for your customized model.

print('Checking for existing uploaded files.')
results = []

# Get the complete list of uploaded files in our subscription.
file_params = {'api-version': api_version}
file_headers = {'api-key': api_key}

r = requests.get(api_base + 'openai/files',
      params=file_params, headers=file_headers)

files = (r.json())["data"]
print(f'Found {len(files)} total uploaded files in the subscription.')

# Enumerate all uploaded files, extracting the file IDs for the
# files with file names that match your training dataset file and
# validation dataset file names.
for item in files:
    if item["filename"] in [training_file_name, validation_file_name, result_file_name]:
        results.append(item["id"])
print(f'Found {len(results)} already uploaded files that match our files')

# Enumerate the file IDs for our files and delete each file.
print(f'Deleting already uploaded files.')
for id in results:
    r = requests.delete(api_base + 'openai/files/' + id,
          params=file_params, headers=file_headers)

Next steps