Léigh i mBéarla Cuir in eagar

Comhroinn trí

Quickstart: Speech to text with the Azure OpenAI Whisper model

  • Alt
Choose a programming language

This quickstart explains how to use the Azure OpenAI Whisper model for speech to text conversion. The Whisper model can transcribe human speech in numerous languages, and it can also translate other languages into English.

The file size limit for the Whisper model is 25 MB. If you need to transcribe a file larger than 25 MB, you can use the Azure AI Speech batch transcription API.

Prerequisites

Set up

Retrieve key and endpoint

To successfully make a call against Azure OpenAI, you need an endpoint and a key.

Variable name Value
AZURE_OPENAI_ENDPOINT The service endpoint can be found in the Keys & Endpoint section when examining your resource from the Azure portal. Alternatively, you can find the endpoint via the Deployments page in Azure AI Foundry portal. An example endpoint is: https://docs-test-001.openai.azure.com/.
AZURE_OPENAI_API_KEY This value can be found in the Keys & Endpoint section when examining your resource from the Azure portal. You can use either KEY1 or KEY2.

Go to your resource in the Azure portal. The Endpoint and Keys can be found in the Resource Management section. Copy your endpoint and access key as you'll need both for authenticating your API calls. You can use either KEY1 or KEY2. Always having two keys allows you to securely rotate and regenerate keys without causing a service disruption.

Screenshot of the overview UI for an Azure OpenAI resource in the Azure portal with the endpoint & access keys location circled in red.

Environment variables

Create and assign persistent environment variables for your key and endpoint.

Tábhachtach

Use API keys with caution. Don't include the API key directly in your code, and never post it publicly. If you use an API key, store it securely in Azure Key Vault. For more information about using API keys securely in your apps, see API keys with Azure Key Vault.

For more information about AI services security, see Authenticate requests to Azure AI services.

CMD 
setx AZURE_OPENAI_API_KEY "REPLACE_WITH_YOUR_KEY_VALUE_HERE"
CMD 
setx AZURE_OPENAI_ENDPOINT "REPLACE_WITH_YOUR_ENDPOINT_HERE"

Create a REST API request and response

In a bash shell, run the following command. You need to replace YourDeploymentName with the deployment name you chose when you deployed the Whisper model. The deployment name isn't necessarily the same as the model name. Entering the model name results in an error unless you chose a deployment name that is identical to the underlying model name.

Bash 
curl $AZURE_OPENAI_ENDPOINT/openai/deployments/YourDeploymentName/audio/transcriptions?api-version=2024-02-01 \
 -H "api-key: $AZURE_OPENAI_API_KEY" \
 -H "Content-Type: multipart/form-data" \
 -F file="@./wikipediaOcelot.wav"

The first line of the preceding command with an example endpoint would appear as follows:

Bash 
curl https://aoai-docs.openai.azure.com/openai/deployments/{YourDeploymentName}/audio/transcriptions?api-version=2024-02-01 \

You can get sample audio files, such as wikipediaOcelot.wav, from the Azure AI Speech SDK repository at GitHub.

Tábhachtach

For production, store and access your credentials using a secure method, such as Azure Key Vault. For more information about credential security, see Azure AI services security.

Output

Bash 
{"text":"The ocelot, Lepardus paradalis, is a small wild cat native to the southwestern United States, Mexico, and Central and South America. This medium-sized cat is characterized by solid black spots and streaks on its coat, round ears, and white neck and undersides. It weighs between 8 and 15.5 kilograms, 18 and 34 pounds, and reaches 40 to 50 centimeters 16 to 20 inches at the shoulders. It was first described by Carl Linnaeus in 1758. Two subspecies are recognized, L. p. paradalis and L. p. mitis. Typically active during twilight and at night, the ocelot tends to be solitary and territorial. It is efficient at climbing, leaping, and swimming. It preys on small terrestrial mammals such as armadillo, opossum, and lagomorphs."}

Prerequisites

Set up

Retrieve key and endpoint

To successfully make a call against Azure OpenAI, you need an endpoint and a key.

Variable name Value
AZURE_OPENAI_ENDPOINT The service endpoint can be found in the Keys & Endpoint section when examining your resource from the Azure portal. Alternatively, you can find the endpoint via the Deployments page in Azure AI Foundry portal. An example endpoint is: https://docs-test-001.openai.azure.com/.
AZURE_OPENAI_API_KEY This value can be found in the Keys & Endpoint section when examining your resource from the Azure portal. You can use either KEY1 or KEY2.

Go to your resource in the Azure portal. The Endpoint and Keys can be found in the Resource Management section. Copy your endpoint and access key as you'll need both for authenticating your API calls. You can use either KEY1 or KEY2. Always having two keys allows you to securely rotate and regenerate keys without causing a service disruption.

Screenshot of the overview UI for an Azure OpenAI resource in the Azure portal with the endpoint & access keys location circled in red.

Environment variables

Create and assign persistent environment variables for your key and endpoint.

Tábhachtach

Use API keys with caution. Don't include the API key directly in your code, and never post it publicly. If you use an API key, store it securely in Azure Key Vault. For more information about using API keys securely in your apps, see API keys with Azure Key Vault.

For more information about AI services security, see Authenticate requests to Azure AI services.

CMD 
setx AZURE_OPENAI_API_KEY "REPLACE_WITH_YOUR_KEY_VALUE_HERE"
CMD 
setx AZURE_OPENAI_ENDPOINT "REPLACE_WITH_YOUR_ENDPOINT_HERE"

Passwordless authentication is recommended

For passwordless authentication, you need to:

  1. Use the @azure/identity package.
  2. Assign the Cognitive Services User role to your user account. This can be done in the Azure portal under Access control (IAM) > Add role assignment.
  3. Sign in with the Azure CLI such as az login.

Create a Python environment

Install the OpenAI Python client library with:

Console 
pip install openai

Create the Python app

  1. Create a new Python file called quickstart.py. Then open it up in your preferred editor or IDE.

  2. Replace the contents of quickstart.py with the following code. Modify the code to add your deployment name:

Python 
    import os
    from openai import AzureOpenAI
        
    client = AzureOpenAI(
        api_key=os.getenv("AZURE_OPENAI_API_KEY"),  
        api_version="2024-02-01",
        azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
    )
    
    deployment_id = "YOUR-DEPLOYMENT-NAME-HERE" #This will correspond to the custom name you chose for your deployment when you deployed a model."
    audio_test_file = "./wikipediaOcelot.wav"
    
    result = client.audio.transcriptions.create(
        file=open(audio_test_file, "rb"),            
        model=deployment_id
    )
    
    print(result)

Run the application using the python command on your quickstart file:

Python 
python quickstart.py

You can get sample audio files, such as wikipediaOcelot.wav, from the Azure AI Speech SDK repository at GitHub.

Tábhachtach

For production, store and access your credentials using a secure method, such as Azure Key Vault. For more information about credential security, see Azure AI services security.

Output

Python 
{"text":"The ocelot, Lepardus paradalis, is a small wild cat native to the southwestern United States, Mexico, and Central and South America. This medium-sized cat is characterized by solid black spots and streaks on its coat, round ears, and white neck and undersides. It weighs between 8 and 15.5 kilograms, 18 and 34 pounds, and reaches 40 to 50 centimeters 16 to 20 inches at the shoulders. It was first described by Carl Linnaeus in 1758. Two subspecies are recognized, L. p. paradalis and L. p. mitis. Typically active during twilight and at night, the ocelot tends to be solitary and territorial. It is efficient at climbing, leaping, and swimming. It preys on small terrestrial mammals such as armadillo, opossum, and lagomorphs."}

Prerequisites

Set up

Retrieve key and endpoint

To successfully make a call against Azure OpenAI, you need an endpoint and a key.

Variable name Value
AZURE_OPENAI_ENDPOINT The service endpoint can be found in the Keys & Endpoint section when examining your resource from the Azure portal. Alternatively, you can find the endpoint via the Deployments page in Azure AI Foundry portal. An example endpoint is: https://docs-test-001.openai.azure.com/.
AZURE_OPENAI_API_KEY This value can be found in the Keys & Endpoint section when examining your resource from the Azure portal. You can use either KEY1 or KEY2.

Go to your resource in the Azure portal. The Endpoint and Keys can be found in the Resource Management section. Copy your endpoint and access key as you'll need both for authenticating your API calls. You can use either KEY1 or KEY2. Always having two keys allows you to securely rotate and regenerate keys without causing a service disruption.

Screenshot of the overview UI for an Azure OpenAI resource in the Azure portal with the endpoint & access keys location circled in red.

Create the .NET app

  1. Create a .NET app using the dotnet new command:

    .NET CLI 
    dotnet new console -n OpenAIWhisper

  2. Change into the directory of the new app:

    .NET CLI 
    cd OpenAIWhisper

  3. Install the Azure.OpenAI client library:

    .NET CLI 
    dotnet add package Azure.AI.OpenAI

Passwordless authentication is more secure than key-based alternatives and is the recommended approach for connecting to Azure services. If you choose to use Passwordless authentication, you'll need to complete the following:

  1. Add the Azure.Identity package.

    .NET CLI 
    dotnet add package Azure.Identity

  2. Assign the Cognitive Services User role to your user account. This can be done in the Azure portal on your OpenAI resource under Access control (IAM) > Add role assignment.

  3. Sign-in to Azure using Visual Studio or the Azure CLI via az login.

Update the app code

  1. Replace the contents of program.cs with the following code and update the placeholder values with your own.

    Nóta

    You can get sample audio files, such as wikipediaOcelot.wav, from the Azure AI Speech SDK repository at GitHub.

    C# 
    using Azure;
using Azure.AI.OpenAI;
using Azure.Identity; // Required for Passwordless auth

var endpoint = new Uri("YOUR_OPENAI_ENDPOINT");
var credentials = new AzureKeyCredential("YOUR_OPENAI_KEY");
// var credentials = new DefaultAzureCredential(); // Use this line for Passwordless auth
var deploymentName = "whisper"; // Default deployment name, update with your own if necessary
var audioFilePath = "YOUR_AUDIO_FILE_PATH";

var openAIClient = new AzureOpenAIClient(endpoint, credentials);

var audioClient = openAIClient.GetAudioClient(deploymentName);

var result = await audioClient.TranscribeAudioAsync(audioFilePath);

Console.WriteLine("Transcribed text:");
foreach (var item in result.Value.Text)
{
    Console.Write(item);
}

Tábhachtach

Use API keys with caution. Don't include the API key directly in your code, and never post it publicly. If you use an API key, store it securely in Azure Key Vault. For more information about using API keys securely in your apps, see API keys with Azure Key Vault.

For more information about AI services security, see Authenticate requests to Azure AI services.

  1. Run the application using the dotnet run command or the run button at the top of Visual Studio:

    .NET CLI 
    dotnet run

    If you are using the sample audio file, you should see the following text printed out in the console:

    text 
    The ocelot, Lepardus paradalis, is a small wild cat native to the southwestern United States, 
Mexico, and Central and South America. This medium-sized cat is characterized by solid 
black spots and streaks on its coat, round ears...

Source code | Package (npm) | Samples

Prerequisites

Microsoft Entra ID prerequisites

For the recommended keyless authentication with Microsoft Entra ID, you need to:

  • Install the Azure CLI used for keyless authentication with Microsoft Entra ID.
  • Assign the Cognitive Services User role to your user account. You can assign roles in the Azure portal under Access control (IAM) > Add role assignment.

Set up

  1. Create a new folder synthesis-quickstart to contain the application and open Visual Studio Code in that folder with the following command:

    shell 
    mkdir synthesis-quickstart && cd synthesis-quickstart

  2. Create the package.json with the following command:

    shell 
    npm init -y

  3. Install the OpenAI client library for JavaScript with:

    Console 
    npm install openai

  4. For the recommended passwordless authentication:

    Console 
    npm install @azure/identity

Retrieve resource information

You need to retrieve the following information to authenticate your application with your Azure OpenAI resource:

Variable name Value
AZURE_OPENAI_ENDPOINT This value can be found in the Keys and Endpoint section when examining your resource from the Azure portal.
AZURE_OPENAI_DEPLOYMENT_NAME This value will correspond to the custom name you chose for your deployment when you deployed a model. This value can be found under Resource Management > Model Deployments in the Azure portal.
OPENAI_API_VERSION Learn more about API Versions.

You can change the version in code or use an environment variable.

Learn more about keyless authentication and setting environment variables.

Rabhadh

To use the recommended keyless authentication with the SDK, make sure that the AZURE_OPENAI_API_KEY environment variable isn't set.

Create a sample application

  1. Create the index.js file with the following code:

    JavaScript 
    const { createReadStream } = require("fs");
const { AzureOpenAI } = require("openai");
const { DefaultAzureCredential, getBearerTokenProvider } = require("@azure/identity");

// You will need to set these environment variables or edit the following values
const audioFilePath = "<audio file path>";
const endpoint = process.env.AZURE_OPENAI_ENDPOINT || "Your endpoint";

// Required Azure OpenAI deployment name and API version
const apiVersion = process.env.OPENAI_API_VERSION || "2024-08-01-preview";
const deploymentName = process.env.AZURE_OPENAI_DEPLOYMENT_NAME || "whisper";

// keyless authentication    
const credential = new DefaultAzureCredential();
const scope = "https://cognitiveservices.azure.com/.default";
const azureADTokenProvider = getBearerTokenProvider(credential, scope);

function getClient() {
  return new AzureOpenAI({
    endpoint,
    azureADTokenProvider,
    apiVersion,
    deployment: deploymentName,
  });
}

export async function main() {
  console.log("== Transcribe Audio Sample ==");

  const client = getClient();
  const result = await client.audio.transcriptions.create({
    model: "",
    file: createReadStream(audioFilePath),
  });

  console.log(`Transcription: ${result.text}`);
}

main().catch((err) => {
  console.error("The sample encountered an error:", err);
});

  2. Sign in to Azure with the following command:

    shell 
    az login

  3. Run the JavaScript file.

    shell 
    node index.js

You can get sample audio files, such as wikipediaOcelot.wav, from the Azure AI Speech SDK repository at GitHub.

Output

JSON 
{"text":"The ocelot, Lepardus paradalis, is a small wild cat native to the southwestern United States, Mexico, and Central and South America. This medium-sized cat is characterized by solid black spots and streaks on its coat, round ears, and white neck and undersides. It weighs between 8 and 15.5 kilograms, 18 and 34 pounds, and reaches 40 to 50 centimeters 16 to 20 inches at the shoulders. It was first described by Carl Linnaeus in 1758. Two subspecies are recognized, L. p. paradalis and L. p. mitis. Typically active during twilight and at night, the ocelot tends to be solitary and territorial. It is efficient at climbing, leaping, and swimming. It preys on small terrestrial mammals such as armadillo, opossum, and lagomorphs."}

Source code | Package (npm) | Samples

Prerequisites

Microsoft Entra ID prerequisites

For the recommended keyless authentication with Microsoft Entra ID, you need to:

  • Install the Azure CLI used for keyless authentication with Microsoft Entra ID.
  • Assign the Cognitive Services User role to your user account. You can assign roles in the Azure portal under Access control (IAM) > Add role assignment.

Set up

  1. Create a new folder whisper-quickstart to contain the application and open Visual Studio Code in that folder with the following command:

    shell 
    mkdir whisper-quickstart && cd whisper-quickstart

  2. Create the package.json with the following command:

    shell 
    npm init -y

  3. Update the package.json to ECMAScript with the following command:

    shell 
    npm pkg set type=module

  4. Install the OpenAI client library for JavaScript with:

    Console 
    npm install openai

  5. For the recommended passwordless authentication:

    Console 
    npm install @azure/identity

Retrieve resource information

You need to retrieve the following information to authenticate your application with your Azure OpenAI resource:

Variable name Value
AZURE_OPENAI_ENDPOINT This value can be found in the Keys and Endpoint section when examining your resource from the Azure portal.
AZURE_OPENAI_DEPLOYMENT_NAME This value will correspond to the custom name you chose for your deployment when you deployed a model. This value can be found under Resource Management > Model Deployments in the Azure portal.
OPENAI_API_VERSION Learn more about API Versions.

You can change the version in code or use an environment variable.

Learn more about keyless authentication and setting environment variables.

Rabhadh

To use the recommended keyless authentication with the SDK, make sure that the AZURE_OPENAI_API_KEY environment variable isn't set.

Create a sample application

  1. Create the index.ts file with the following code:

    TypeScript 
    import { createReadStream } from "fs";
import { AzureOpenAI } from "openai";
import { DefaultAzureCredential, getBearerTokenProvider } from "@azure/identity";

// You will need to set these environment variables or edit the following values
const audioFilePath = "<audio file path>";
const endpoint = process.env.AZURE_OPENAI_ENDPOINT || "Your endpoint";

// Required Azure OpenAI deployment name and API version
const apiVersion = process.env.OPENAI_API_VERSION || "2024-08-01-preview";
const deploymentName = process.env.AZURE_OPENAI_DEPLOYMENT_NAME || "whisper";

// keyless authentication    
const credential = new DefaultAzureCredential();
const scope = "https://cognitiveservices.azure.com/.default";
const azureADTokenProvider = getBearerTokenProvider(credential, scope);

function getClient(): AzureOpenAI {
  return new AzureOpenAI({
    endpoint,
    azureADTokenProvider,
    apiVersion,
    deployment: deploymentName,
  });
}

export async function main() {
  console.log("== Transcribe Audio Sample ==");

  const client = getClient();
  const result = await client.audio.transcriptions.create({
    model: "",
    file: createReadStream(audioFilePath),
  });

  console.log(`Transcription: ${result.text}`);
}

main().catch((err) => {
  console.error("The sample encountered an error:", err);
});

  2. Create the tsconfig.json file to transpile the TypeScript code and copy the following code for ECMAScript.

    JSON 
    {
    "compilerOptions": {
      "module": "NodeNext",
      "target": "ES2022", // Supports top-level await
      "moduleResolution": "NodeNext",
      "skipLibCheck": true, // Avoid type errors from node_modules
      "strict": true // Enable strict type-checking options
    },
    "include": ["*.ts"]
}

  3. Transpile from TypeScript to JavaScript.

    shell 
    tsc

  4. Sign in to Azure with the following command:

    shell 
    az login

  5. Run the code with the following command:

    shell 
    node index.js

You can get sample audio files, such as wikipediaOcelot.wav, from the Azure AI Speech SDK repository at GitHub.

Tábhachtach

Use API keys with caution. Don't include the API key directly in your code, and never post it publicly. If you use an API key, store it securely in Azure Key Vault. For more information about using API keys securely in your apps, see API keys with Azure Key Vault.

For more information about AI services security, see Authenticate requests to Azure AI services.

Output

JSON 
{"text":"The ocelot, Lepardus paradalis, is a small wild cat native to the southwestern United States, Mexico, and Central and South America. This medium-sized cat is characterized by solid black spots and streaks on its coat, round ears, and white neck and undersides. It weighs between 8 and 15.5 kilograms, 18 and 34 pounds, and reaches 40 to 50 centimeters 16 to 20 inches at the shoulders. It was first described by Carl Linnaeus in 1758. Two subspecies are recognized, L. p. paradalis and L. p. mitis. Typically active during twilight and at night, the ocelot tends to be solitary and territorial. It is efficient at climbing, leaping, and swimming. It preys on small terrestrial mammals such as armadillo, opossum, and lagomorphs."}

Prerequisites

Set up

Retrieve key and endpoint

To successfully make a call against Azure OpenAI, you need an endpoint and a key.

Variable name Value
AZURE_OPENAI_ENDPOINT The service endpoint can be found in the Keys & Endpoint section when examining your resource from the Azure portal. Alternatively, you can find the endpoint via the Deployments page in Azure AI Foundry portal. An example endpoint is: https://docs-test-001.openai.azure.com/.
AZURE_OPENAI_API_KEY This value can be found in the Keys & Endpoint section when examining your resource from the Azure portal. You can use either KEY1 or KEY2.

Go to your resource in the Azure portal. The Endpoint and Keys can be found in the Resource Management section. Copy your endpoint and access key as you'll need both for authenticating your API calls. You can use either KEY1 or KEY2. Always having two keys allows you to securely rotate and regenerate keys without causing a service disruption.

Screenshot of the overview UI for an Azure OpenAI resource in the Azure portal with the endpoint & access keys location circled in red.

Environment variables

Create and assign persistent environment variables for your key and endpoint.

Tábhachtach

Use API keys with caution. Don't include the API key directly in your code, and never post it publicly. If you use an API key, store it securely in Azure Key Vault. For more information about using API keys securely in your apps, see API keys with Azure Key Vault.

For more information about AI services security, see Authenticate requests to Azure AI services.

CMD 
setx AZURE_OPENAI_API_KEY "REPLACE_WITH_YOUR_KEY_VALUE_HERE"
CMD 
setx AZURE_OPENAI_ENDPOINT "REPLACE_WITH_YOUR_ENDPOINT_HERE"

Create a PowerShell app

Run the following command. You need to replace YourDeploymentName with the deployment name you chose when you deployed the Whisper model. The deployment name isn't necessarily the same as the model name. Entering the model name results in an error unless you chose a deployment name that is identical to the underlying model name.

PowerShell 
# Azure OpenAI metadata variables
$openai = @{
    api_key     = $Env:AZURE_OPENAI_API_KEY
    api_base    = $Env:AZURE_OPENAI_ENDPOINT # your endpoint should look like the following https://YOUR_RESOURCE_NAME.openai.azure.com/
    api_version = '2024-02-01' # this may change in the future
    name        = 'YourDeploymentName' #This will correspond to the custom name you chose for your deployment when you deployed a model.
}

# Header for authentication
$headers = [ordered]@{
    'api-key' = $openai.api_key
}

$form = @{ file = get-item -path './wikipediaOcelot.wav' }

# Send a completion call to generate an answer
$url = "$($openai.api_base)/openai/deployments/$($openai.name)/audio/transcriptions?api-version=$($openai.api_version)"

$response = Invoke-RestMethod -Uri $url -Headers $headers -Form $form -Method Post -ContentType 'multipart/form-data'
return $response.text

You can get sample audio files, such as wikipediaOcelot.wav, from the Azure AI Speech SDK repository at GitHub.

Tábhachtach

For production, store and access your credentials using a secure method, such as The PowerShell Secret Management with Azure Key Vault. For more information about credential security, see Azure AI services security.

Output

text 
The ocelot, Lepardus paradalis, is a small wild cat native to the southwestern United States, Mexico, and Central and South America. This medium-sized cat is characterized by solid black spots and streaks on its coat, round ears, and white neck and undersides. It weighs between 8 and 15.5 kilograms, 18 and 34 pounds, and reaches 40 to 50 centimeters 16 to 20 inches at the shoulders. It was first described by Carl Linnaeus in 1758. Two subspecies are recognized, L. p. paradalis and L. p. mitis. Typically active during twilight and at night, the ocelot tends to be solitary and territorial. It is efficient at climbing, leaping, and swimming. It preys on small terrestrial mammals such as armadillo, opossum, and lagomorphs.

Clean up resources

If you want to clean up and remove an Azure OpenAI resource, you can delete the resource. Before deleting the resource, you must first delete any deployed models.

Next steps