Quickstart: Hear and speak with chat models in the Azure AI Foundry portal chat playground
In the chat playground in Azure AI Foundry portal, you can use speech to text and text to speech features to interact with chat models. You can try the same model that you use for text-based chat in a speech-based chat. It's just another way to interact with the model.
In this quickstart, you use Azure OpenAI Service and Azure AI Speech to:
- Speak to the assistant via speech to text.
- Hear the assistant's response via text to speech.
The speech to text and text to speech features can be used together or separately in the Azure AI Foundry portal chat playground. You can use the playground to test your chat model before deploying it.
Prerequisites
- An Azure subscription - Create one for free.
- An Azure AI Foundry project.
- A deployed Azure OpenAI chat model. This guide is tested with a
gpt-4o-minimodel.
Configure the chat playground
Before you can start a chat session, you need to configure the chat playground to use the speech to text and text to speech features.
Sign in to the Azure AI Foundry portal.
Go to your Azure AI Foundry project. If you need to create a project, see Create an Azure AI Foundry project.
Select Playgrounds from the left pane and then select a playground to use. In this example, select Try the chat playground.
Select your deployed chat model from the Deployment dropdown.
Select the Chat capabilities button.
Note
You should also see the options to select the microphone or speaker buttons. If you select either of these buttons, but didn't yet enable speech to text or text to speech, you're prompted to enable them in Chat capabilities.
On the Chat capabilities page, select the box to acknowledge that usage of the speech feature incurs extra costs. For more information, see Azure AI Speech pricing.
Select Enable speech to text and Enable text to speech.
Select the language locale and voice you want to use for speaking and hearing. The list of available voices depends on the locale that you select.
Optionally, you can try the voice before you return to the chat session. Enter some sample text and select Play to
Select Save.
Start a chat session
In this chat session, you use both speech to text and text to speech. You use the speech to text feature to speak to the assistant, and the text to speech feature to hear the assistant's response.
Complete the steps in the Configure the playground section. To complete this quickstart, you need to enable the speech to text and text to speech features.
Select the microphone button and speak to the assistant. For example, you can say "Do you know where I can get an Xbox".
Select the send button (right arrow) to send your message to the assistant. The assistant's response is displayed in the chat session pane.
Note
If the speaker button is turned on, you hear the assistant's response. If the speaker button is turned off, you don't hear the assistant's response, but the response is still displayed in the chat session pane.
You can change the system prompt to change the assistant's response format or style.
For example, enter:
"You're an AI assistant that helps people find information. Answers shouldn't be longer than 20 words because you are on a phone. You could use 'um' or 'let me see' to make it more natural and add some disfluency."Say again: "Do you know where I can get an Xbox". The response is shown in the chat session pane. Since the speaker button is turned on, you also hear the response.
Clean up resources
To avoid incurring unnecessary Azure costs, you should delete the resources you created in this quickstart if they're no longer needed. To manage resources, you can use the Azure portal.