Cuir in eagar

Comhroinn trí

Add a professional voice training dataset

  • Alt

When you're ready to create a custom text to speech voice for your application, the first step is to gather audio recordings and associated scripts to start training the voice model. For details on recording voice samples, see the tutorial. The Speech service uses this data to create a unique voice tuned to match the voice in the recordings. After you've trained the voice, you can start synthesizing speech in your applications.

All data you upload must meet the requirements for the data type that you choose. It's important to correctly format your data before it's uploaded, which ensures the data will be accurately processed by the Speech service. To confirm that your data is correctly formatted, see Training data types.

Note

  • Standard subscription (S0) users can upload five data files simultaneously. If you reach the limit, wait until at least one of your data files finishes importing. Then try again.
  • The maximum number of data files allowed to be imported per subscription is 500 .zip files for standard subscription (S0) users. Please see out Speech service quotas and limits for more details.

Upload your data

When you're ready to upload your data, go to the Prepare training data tab to add your first training set and upload data. A training set is a set of audio utterances and their mapping scripts used for training a voice model. You can use a training set to organize your training data. The service checks data readiness per each training set. You can import multiple data to a training set.

To upload training data, follow these steps:

  1. Sign in to the Speech Studio.
  2. Select Custom voice > Your project name > Prepare training data > Upload data.
  3. In the Upload data wizard, choose a data type and then select Next.
  4. Select local files from your computer or enter the Azure Blob storage URL to upload data.
  5. Under Specify the target training set, select an existing training set or create a new one. If you created a new training set, make sure it's selected in the drop-down list before you continue.
  6. Select Next.
  7. Enter a name and description for your data and then select Next.
  8. Review the upload details, and select Submit.

Note

Duplicate IDs are not accepted. Utterances with the same ID will be removed.

Duplicate audio names are removed from the training. Make sure the data you select don't contain the same audio names within the .zip file or across multiple .zip files. If utterance IDs (either in audio or script files) are duplicates, they're rejected.

Data files are automatically validated when you select Submit. Data validation includes series of checks on the audio files to verify their file format, size, and sampling rate. If there are any errors, fix them and submit again.

After you upload the data, you can check the details in the training set detail view. On the detail page, you can further check the pronunciation issue and the noise level for each of your data. The pronunciation score at the sentence level ranges from 0-100. A score below 70 normally indicates a speech error or script mismatch. Utterances with an overall score lower than 70 will be rejected. A heavy accent can reduce your pronunciation score and affect the generated digital voice.

Resolve data issues online

After upload, you can check the data details of the training set. Before continuing to train your voice model, you should try to resolve any data issues.

You can identify and resolve data issues per utterance in Speech Studio.

  1. On the detail page, go to the Accepted data or Rejected data page. Select individual utterances you want to change, then select Edit.

    Screenshot of selecting edit button on the accepted data or rejected data details page.

    You can choose which data issues to be displayed based on your criteria.

    Screenshot of choosing which data issues to be displayed.

  2. Edit window will be displayed.

    Screenshot of displaying Edit transcript and recording file window.

  3. Update transcript or recording file according to issue description on the edit window.

    You can edit transcript in the text box, then select Done

    Screenshot of selecting Done button on the Edit transcript and recording file window.

    If you need to update recording file, select Update recording file, then upload the fixed recording file (.wav).

    Screenshot that shows how to upload recording file on the Edit transcript and recording file window.

  4. After you've made changes to your data, you need to check the data quality by clicking Analyze data before using this dataset for training.

    You can't select this training set for training model before the analysis is complete.

    Screenshot of selecting Analyze data on Data details page.

    You can also delete utterances with issues by selecting them and clicking Delete.

Typical data issues

The issues are divided into three types. Refer to the following tables to check the respective types of errors.

Auto-rejected

Data with these errors won't be used for training. Imported data with errors will be ignored, so you don't need to delete them. You can fix these data errors online or upload the corrected data again for training.

Category Name Description
Script Invalid separator You must separate the utterance ID and the script content with a Tab character.
Script Invalid script ID The script line ID must be numeric.
Script Duplicated script Each line of the script content must be unique. The line is duplicated with {}.
Script Script too long The script must be less than 1,000 characters.
Script No matching audio The ID of each utterance (each line of the script file) must match the audio ID.
Script No valid script No valid script is found in this dataset. Fix the script lines that appear in the detailed issue list.
Audio No matching script No audio files match the script ID. The name of the .wav files must match with the IDs in the script file.
Audio Invalid audio format The audio format of the .wav files is invalid. Check the .wav file format by using an audio tool like SoX.
Audio Low sampling rate The sampling rate of the .wav files can't be lower than 16 KHz.
Audio Too long audio Audio duration is longer than 30 seconds. Split the long audio into multiple files. It's a good idea to make utterances shorter than 15 seconds.
Audio No valid audio No valid audio is found in this dataset. Check your audio data and upload again.
Mismatch Low scored utterance Sentence-level pronunciation score is lower than 70. Review the script and the audio content to make sure they match.

Auto-fixed

The following errors are fixed automatically, but you should review and confirm the fixes are made correctly.

Category Name Description
Mismatch Silence auto fixed The start silence is detected to be shorter than 100 ms, and has been extended to 100 ms automatically. Download the normalized dataset and review it.
Mismatch Silence auto fixed The end silence is detected to be shorter than 100 ms, and has been extended to 100 ms automatically. Download the normalized dataset and review it.
Script Text auto normalized Text is automatically normalized for digits, symbols, and abbreviations. Review the script and audio to make sure they match.

Manual check required

Unresolved errors listed in the next table affect the quality of training, but data with these errors won't be excluded during training. For higher-quality training, it's a good idea to fix these errors manually.

Category Name Description
Script Non-normalized text This script contains symbols. Normalize the symbols to match the audio. For example, normalize / to slash.
Script Not enough question utterances At least 10 percent of the total utterances should be question sentences. This helps the voice model properly express a questioning tone.
Script Not enough exclamation utterances At least 10 percent of the total utterances should be exclamation sentences. This helps the voice model properly express an excited tone.
Script No valid end punctuation Add one of the following at the end of the line: full stop (half-width '.' or full-width '。'), exclamation point (half-width '!' or full-width '！' ), or question mark ( half-width '?' or full-width '？').
Audio Low sampling rate for neural voice It's recommended that the sampling rate of your .wav files should be 24 KHz or higher for creating neural voices. If it's lower, it will be automatically raised to 24 KHz.
Volume Overall volume too low Volume shouldn't be lower than -18 dB (10 percent of max volume). Control the volume average level within proper range during the sample recording or data preparation.
Volume Volume overflow Overflowing volume is detected at {}s. Adjust the recording equipment to avoid the volume overflow at its peak value.
Volume Start silence issue The first 100 ms of silence isn't clean. Reduce the recording noise floor level, and leave the first 100 ms at the start silent.
Volume End silence issue The last 100 ms of silence isn't clean. Reduce the recording noise floor level, and leave the last 100 ms at the end silent.
Mismatch Low scored words Review the script and the audio content to make sure they match, and control the noise floor level. Reduce the length of long silence, or split the audio into multiple utterances if it's too long.
Mismatch Start silence issue Extra audio was heard before the first word. Review the script and the audio content to make sure they match, control the noise floor level, and make the first 100 ms silent.
Mismatch End silence issue Extra audio was heard after the last word. Review the script and the audio content to make sure they match, control the noise floor level, and make the last 100 ms silent.
Mismatch Low signal-noise ratio Audio SNR level is lower than 20 dB. At least 35 dB is recommended.
Mismatch No score available Failed to recognize speech content in this audio. Check the audio and the script content to make sure the audio is valid, and matches the script.

Next steps

Train the professional voice

You need a training dataset to create a professional voice. A training dataset includes audio and script files. The audio files are recordings of the voice talent reading the script files. The script files are the text of the audio files.

In this article, you create a training set and get its resource ID. Then, using the resource ID, you can upload a set of audio and script files.

Create a training set

To create a training set, use the TrainingSets_Create operation of the custom voice API. Construct the request body according to the following instructions:

  • Set the required projectId property. See create a project.
  • Set the required voiceKind property to Male or Female. The kind can't be changed later.
  • Set the required locale property. This should be the locale of the training set data. The locale of the training set should be the same as the locale of the consent statement. The locale can't be changed later. You can find the text to speech locale list here.
  • Optionally, set the description property for the training set description. The training set description can be changed later.

Make an HTTP PUT request using the URI as shown in the following TrainingSets_Create example.

  • Replace YourResourceKey with your Speech resource key.
  • Replace YourResourceRegion with your Speech resource region.
  • Replace JessicaTrainingSetId with a training set ID of your choice. The case sensitive ID will be used in the training set's URI and can't be changed later.
curl -v -X PUT -H "Ocp-Apim-Subscription-Key: YourResourceKey" -H "Content-Type: application/json" -d '{
  "description": "300 sentences Jessica data in general style.",
  "projectId": "ProjectId",
  "locale": "en-US",
  "voiceKind": "Female"
} '  "https://YourResourceRegion.api.cognitive.microsoft.com/customvoice/trainingsets/JessicaTrainingSetId?api-version=2024-02-01-preview"

You should receive a response body in the following format:

{
  "id": "JessicaTrainingSetId",
  "description": "300 sentences Jessica data in general style.",
  "projectId": "ProjectId",
  "locale": "en-US",
  "voiceKind": "Female",
  "status": "Succeeded",
  "createdDateTime": "2023-04-01T05:30:00.000Z",
  "lastActionDateTime": "2023-04-02T10:15:30.000Z"
}

Upload training set data

To upload a training set of audio and scripts, use the TrainingSets_UploadData operation of the custom voice API.

Before calling this API, please store recording and script files in Azure Blob. In the example below, recording files are https://contoso.blob.core.windows.net/voicecontainer/jessica300/*.wav, script files are https://contoso.blob.core.windows.net/voicecontainer/jessica300/*.txt.

Construct the request body according to the following instructions:

  • Set the required kind property to AudioAndScript. The kind determines the type of training set.
  • Set the required audios property. Within the audios property, set the following properties:
    • Set the required containerUrl property to the URL of the Azure Blob Storage container that contains the audio files. Use shared access signatures (SAS) for a container with both read and list permissions.
    • Set the required extensions property to the extensions of the audio files.
    • Optionally, set the prefix property to set a prefix for the blob name.
  • Set the required scripts property. Within the scripts property, set the following properties:
    • Set the required containerUrl property to the URL of the Azure Blob Storage container that contains the script files. Use shared access signatures (SAS) for a container with both read and list permissions.
    • Set the required extensions property to the extensions of the script files.
    • Optionally, set the prefix property to set a prefix for the blob name.

Make an HTTP POST request using the URI as shown in the following TrainingSets_UploadData example.

  • Replace YourResourceKey with your Speech resource key.
  • Replace YourResourceRegion with your Speech resource region.
  • Replace JessicaTrainingSetId if you specified a different training set ID in the previous step.
curl -v -X POST -H "Ocp-Apim-Subscription-Key: YourResourceKey" -H "Content-Type: application/json" -d '{
  "kind": "AudioAndScript",
  "audios": {
    "containerUrl": "https://contoso.blob.core.windows.net/voicecontainer?mySasToken",
    "prefix": "jessica300/",
    "extensions": [
      ".wav"
    ]
  },
  "scripts": {
    "containerUrl": "https://contoso.blob.core.windows.net/voicecontainer?mySasToken",
    "prefix": "jessica300/",
    "extensions": [
      ".txt"
    ]
  }
} '  "https://YourResourceRegion.api.cognitive.microsoft.com/customvoice/trainingsets/JessicaTrainingSetId:upload?api-version=2024-02-01-preview"

The response header contains the Operation-Location property. Use this URI to get details about the TrainingSets_UploadData operation. Here's an example of the response header:

Operation-Location: https://eastus.api.cognitive.microsoft.com/customvoice/operations/284b7e37-f42d-4054-8fa9-08523c3de345?api-version=2024-02-01-preview
Operation-Id: 284b7e37-f42d-4054-8fa9-08523c3de345

Next steps

Train the professional voice