Get started with the Azure Communication Services Call Automation OpenAI sample
The Azure Communication Services Call Automation OpenAI sample demonstrates how you can use Call Automation SDK and the recently announced public preview integration with Azure AI services to build intelligent virtual assistants.
In this sample, we'll cover off what this sample does and what you need as pre-requisites before we run this sample locally on your machine.
Download code
Find the project for this sample on GitHub. You can download this code and run it locally to try it for yourself.
Overview
This sample is a server-side application that helps you create a virtual assistant capable of handling calls using Call Automation and responding to customers using the newly announced integration with Azure AI services to provide AI capabilities such as Text-to-Speech and Speech-to-Text along with smart responses provided by Azure OpenAI.
This Azure Communication Services Call Automation AI sample demonstrates how to use the Call Automation SDK to answer an inbound call, recognizes user voice input using Call Automation recognize API with support for Speech-to-Text. Once input is recognized, it sends that information to OpenAI for an answer and plays the answer provided back by OpenAI to the caller using Call Automation play API with support for Text-to-Speech.
Prerequisites
- Create an Azure account with an active subscription. For details, see Create an account for free
- Create an Azure Communication Services resource. For details, see Create an Azure Communication Resource. You need to record your resource connection string for this sample.
- A Calling-enabled telephone number. Get a phone number.
- Azure Dev Tunnels CLI. For details, see Enable dev tunnel
- Create an Azure AI services resource. For details, see Create an Azure AI services Resource
- An Azure OpenAI Resource and Deployed Model. See instructions.
Setup instructions
Before running this sample, you need to set up the resources mentioned in the 'Prerequisites' section with the following configuration updates:
1. Setup and host your Azure DevTunnel
Azure DevTunnels is an Azure service that enables you to share local web services hosted on the internet. Use the commands below to connect your local development environment to the public internet. This creates a tunnel with a persistent endpoint URL and which allows anonymous access. We'll then use this endpoint to notify your application of calling events from the Azure Communication Services Call Automation service.
devtunnel create --allow-anonymous
devtunnel port create -p 5165
devtunnel host
2. Add a Managed Identity to the Azure Communication Services Resource that connects to the Azure AI services resource
Follow the instructions in this documentation.
3. Add the required API Keys and endpoints
Open the appsettings.json file to configure the following settings:
DevTunnelUri
: your dev tunnel endpointCognitiveServiceEndpoint
: The Azure AI services endpointAcsConnectionString
: Azure Communication Service resource's connection string.AzureOpenAIServiceKey
: Azure OpenAI Service KeyAzureOpenAIServiceEndpoint
: Azure OpenAI Service EndpointAzureOpenAIDeploymentModelName
: Azure OpenAI Model name
Run the application
- Azure DevTunnel: Ensure your AzureDevTunnel URI is active and points to the correct port of your localhost application
- Run
dotnet run
to build and run the sample application - Register an Event Grid Webhook for the IncomingCall Event that points to your DevTunnel URI. Instructions here.
Once that's completed, you should have a running application. The best way to test this sample is to place a call to your Azure Communication Services phone number and talk to your intelligent agent.
Next steps
- Learn more about Call Automation.
- Learn more about playing custom messages.
- Learn more about recognizing user input.
Download code
Find the project for this sample on GitHub. You can download this code and run it locally to try it for yourself.
Overview
This sample is a server-side application that helps you create a virtual assistant capable of handling calls using Call Automation and responding to customers using the newly announced integration with Azure AI services to provide AI capabilities such as Text-to-Speech and Speech-to-Text along with smart responses provided by Azure OpenAI.
This Azure Communication Services Call Automation AI sample demonstrates how to use the Call Automation SDK to answer an inbound call. It recognizes user voice input using Call Automation recognize API with support for Speech-to-Text. Once input is recognized, it sends that information to OpenAI for an answer and plays the answer provided back by OpenAI to the caller using Call Automation play API with support for Text-to-Speech.
Prerequisites
- An Azure account with an active subscription. Create an account for free.
- A deployed Communication Services resource. Create a Communication Services resource.
- A phone number in your Azure Communication Services resource that can make outbound calls. NB: phone numbers aren't available in free subscriptions.
- Java Development Kit (JDK) Microsoft.OpenJDK.17
- Apache Maven
- Create and host an Azure Dev Tunnel. Instructions here
- Create an Azure AI services resource. For details, see Create an Azure AI services Resource.
- An Azure OpenAI Resource and Deployed Model. See instructions.
Before running the sample for the first time
Open the application.yml file in the resources folder to configure the following settings
connectionstring
: Azure Communication Service resource's connection string.basecallbackuri
: Base url of the app. For local development use dev tunnel url.cognitiveServicesUrl
: The Azure AI services endpointazureOpenAiServiceKey
: Azure OpenAI Service KeyazureOpenAiServiceEndpoint
: Azure OpenAI Service EndpointopenAiModelName
: Azure OpenAI Model name
Setup and host your Azure DevTunnel
Azure DevTunnels is an Azure service that enables you to share local web services hosted on the internet. Use the commands provided to connect your local development environment to the public internet. This creates a tunnel with a persistent endpoint URL and which allows anonymous access. We'll then use this endpoint to notify your application of calling events from the Azure Communication Services Call Automation service.
devtunnel create --allow-anonymous
devtunnel port create -p 8080
devtunnel host
Run the application
- Navigate to the directory containing the pom.xml file and use the following mvn commands:
- Compile the application: mvn compile
- Build the package: mvn package
- Execute the app: mvn exec:java
- Access the Swagger UI at http://localhost:8080/swagger-ui.html
- Try the GET /outboundCall to run the Sample Application
Once that's completed, you should have a running application. The best way to test this sample is to place a call to your Azure Communication Services phone number and talk to your intelligent agent.
Next steps
- Learn more about Call Automation.
- Learn more about playing custom messages.
- Learn more about recognizing user input.
Download code
Find the project for this sample on GitHub. You can download this code and run it locally to try it for yourself.
Overview
This sample is a server-side application that helps you create a virtual assistant capable of handling calls using Call Automation and responding to customers using the newly announced integration with Azure AI services to provide AI capabilities such as Text-to-Speech and Speech-to-Text along with smart responses provided by Azure OpenAI.
This Azure Communication Services Call Automation AI sample demonstrates how to use the Call Automation SDK to answer an inbound call, recognizes user voice input using Call Automation recognize API with support for Speech-to-Text. Once input is recognized, it sends that information to OpenAI for an answer and plays the answer that is provided back by OpenAI to the caller using Call Automation play API with support for Text-to-Speech.
Prerequisites
- Create an Azure account with an active subscription. For details, see Create an account for free
- Create an Azure Communication Services resource. For details, see Create an Azure Communication Resource. You need to record your resource connection string for this sample.
- A Calling-enabled telephone number. Get a phone number. -- Node.js installed
- Azure Dev Tunnels CLI. For details, see Enable dev tunnel
- Create an Azure AI Multi service resource. For details, see Create an Azure AI services Resource
- An Azure OpenAI Resource and Deployed Model. See instructions.
Setup instructions
Before running this sample, you need to set up the resources mentioned in the 'Prerequisites' section with the following configuration updates:
1. Setup and host your Azure DevTunnel
Azure DevTunnels is an Azure service that enables you to share local web services hosted on the internet. Use the commands provided in this document to connect your local development environment to the public internet. This creates a tunnel with a persistent endpoint URL and which allows anonymous access. We'll use this endpoint to notify your application of calling events from the ACS Call Automation service.
devtunnel create --allow-anonymous
devtunnel port create -p 8080
devtunnel host
2. Add a Managed Identity to the Azure Communication Services Resource that connects to the Azure AI services resource
Follow the instructions in this documentation.
3. Add the required API Keys and endpoints
Open the .env
file to configure the following settings
CONNECTION_STRING
: Azure Communication Service resource's connection string.CALLBACK_URI
: Base url of the app. (For local development replace the dev tunnel url)COGNITIVE_SERVICE_ENDPOINT
: Azure AI Service endpointAZURE_OPENAI_SERVICE_KEY
: Azure OpenAI service keyAZURE_OPENAI_SERVICE_ENDPOINT
: Azure OpenAI endpointAZURE_OPENAI_DEPLOYMENT_MODEL_NAME
: Azure OpenAI deployment nameAGENT_PHONE_NUMBER
: Agent phone number to transfer the call to resolve queries
Run the application
- Open a new PowerShell window, cd into the
callautomation-openai-sample
folder and runnpm run dev
- Browser should pop up with the below page. If not, navigate it to
http://localhost:8080/
- Register an Event Grid Webhook for the IncomingCall Event that points to your DevTunnel URI. Instructions here.
Once that's completed, you should have a running application. The best way to test this sample is to place a call to your Azure Communication Services phone number and talk to your intelligent agent.
Next steps
- Learn more about Call Automation.
- Learn more about playing custom messages.
- Learn more about recognizing user input.
Download code
Find the project for this sample on GitHub. You can download this code and run it locally to try it for yourself.
Overview
This sample is a server-side application that helps you create a virtual assistant capable of handling calls using Call Automation. It also enables your assistant to respond to customers using Azure AI services, which provide AI capabilities such as Text-to-Speech and Speech-to-Text, as well as smart responses provided by Azure OpenAI.
This Azure Communication Services Call Automation AI sample demonstrates how to use the Call Automation SDK to answer an inbound call, recognizes user voice input using Call Automation recognize API with support for Speech-to-Text. When the system recognizes the input, it sends the information to OpenAI for an answer and uses the Call Automation play API with support for Text-to-Speech to play back the answer provided by OpenAI to the caller.
Prerequisites
- Create an Azure account with an active subscription. For details, see Create an account for free.
- Create an Azure Communication Services resource. For details, see Create an Azure Communication Resource. You need to record your resource connection string for this sample.
- A Calling-enabled telephone number. Get a phone number.
- Azure Dev Tunnels CLI. For details, see Enable dev tunnel.
- Create an Azure AI Multi service resource. For details, see Create an Azure AI services Resource.
- An Azure OpenAI Resource and Deployed Model. See instructions.
- Create and host an Azure Dev Tunnel. Instructions here.
- Python 3.7 or above (please be aware that currently version 3.12 does not have support for OpenAI libraries).
Setup instructions
Before running this sample, you need to set up the resources mentioned in the 'Prerequisites' section with the following configuration updates:
1. Setup Python environment
Create and activate python virtual environment and install required packages using following command
pip install -r requirements.txt
2. Setup and host your Azure DevTunnel
Azure DevTunnels is an Azure service that enables you to share local web services hosted on the internet. Use the commands provided to connect your local development environment to the public internet. This process creates a tunnel with a persistent endpoint URL and allows anonymous access. We'll use this endpoint to notify your application of calling events from the ACS Call Automation service.
devtunnel create --allow-anonymous
devtunnel port create -p 8080
devtunnel host
3. Add a Managed Identity to the Azure Communication Services Resource that connects to the Azure AI services resource
Follow the instructions in this documentation.
4. Add the required API Keys and endpoints
Open main.py
file to configure the following settings
-
CALLBACK_URI_HOST
: your dev tunnel endpoint
-
COGNITIVE_SERVICE_ENDPOINT
: The Azure AI Services endpoint
-
ACS_CONNECTION_STRING
: Azure Communication Service resource's connection string.
-
AZURE_OPENAI_SERVICE_KEY
: Azure OpenAI Service Key
-
AZURE_OPENAI_SERVICE_ENDPOINT
: Azure OpenAI Service Endpoint
-
AZURE_OPENAI_DEPLOYMENT_MODEL_NAME
: Azure OpenAI Model name
-
AGENT_PHONE_NUMBER
: Agent Phone Number to transfer call
Run the application
- Navigate to
callautomation-openai-sample
folder and runmain.py
in debug mode or use commandpython ./main.py
to run it from PowerShell, Command Prompt or Unix Terminal - Browser should pop up with the below page. If not, navigate it to
http://localhost:8080/
or your dev tunnel url. - Register an Event Grid Webhook for the IncomingCall Event that points to your DevTunnel URI. Instructions here.
Once you complete that, your application should be up and running. The best way to test this sample is to place a call to your Azure Communication Services phone number and talk to your intelligent agent.
Next steps
- Learn more about Call Automation.
- Learn more about playing custom messages.
- Learn more about recognizing user input.