Edit

Share via

Get started with the Azure Communication Services Call Automation OpenAI sample

The Azure Communication Services Call Automation OpenAI sample demonstrates how you can use the Call Automation SDK and the recently announced public preview integration with Azure AI services to build intelligent virtual assistants.

In this article, we cover what this sample does and what you need as prerequisites before we run this sample locally on your machine.

Download code

Find the project for this sample on GitHub. You can download this code and run it locally to try it for yourself.

Overview

This sample is a server-side application that helps you create a virtual assistant that can handle calls by using Call Automation. The virtual assistant responds to customers by using the newly announced integration with Azure AI services to provide AI capabilities such as text to speech and speech to text, along with smart responses provided by Azure OpenAI.

This Azure Communication Services Call Automation AI sample demonstrates how to use the Call Automation SDK to answer an inbound call. It recognizes user voice input by using Call Automation to recognize the API with support for speech to text. When the system recognizes the input, it sends that information to OpenAI for an answer. It uses the Call Automation Play API with support for text to speech to play back the answer provided by OpenAI to the caller.

Prerequisites

Setup instructions

Before you run this sample, you need to set up the resources mentioned in the preceding section with the following configuration updates.

Step 1: Set up and host your Azure dev tunnel

With dev tunnels, you can share local web services that are hosted on the internet. Use the following commands to connect your local development environment to the public internet. This process creates a tunnel with a persistent endpoint URL and allows anonymous access. We use this endpoint to notify your application of calling events from Call Automation.

devtunnel create --allow-anonymous
devtunnel port create -p 5165
devtunnel host

Step 2: Add a managed identity to the Azure Communication Services resource that connects to the Azure AI services resource

Follow the instructions in Connect Azure Communication Services with Azure AI services.

Step 3: Add the required API keys and endpoints

Open the appsettings.json file to configure the following settings:

  • DevTunnelUri: Your dev tunnel endpoint.
  • CognitiveServiceEndpoint: Azure AI services endpoint.
  • AcsConnectionString: Azure Communication Services resource connection string.
  • AzureOpenAIServiceKey: Azure OpenAI service key.
  • AzureOpenAIServiceEndpoint: Azure OpenAI service endpoint.
  • AzureOpenAIDeploymentModelName: Azure OpenAI model name.

Run the application

  1. Ensure that your AzureDevTunnel URI is active and points to the correct port of your local host application.
  2. Run dotnet run to build and run the sample application.
  3. Register an Azure Event Grid webhook for the IncomingCall event that points to your DevTunnel URI. For more information, see Incoming call concepts.

Now you have a running application. The best way to test this sample is to place a call to your Azure Communication Services phone number and talk to your intelligent agent.

Related content

Download code

Find the project for this sample on GitHub. You can download this code and run it locally to try it for yourself.

Overview

This sample is a server-side application that helps you create a virtual assistant that can handle calls by using Call Automation. The virtual assistant responds to customers by using the newly announced integration with Azure AI services to provide AI capabilities such as text to speech and speech to text, along with smart responses provided by Azure OpenAI.

This Azure Communication Services Call Automation AI sample demonstrates how to use the Call Automation SDK to answer an inbound call. It recognizes user voice input by using Call Automation to recognize the API with support for speech to text. When the system recognizes the input, it sends that information to OpenAI for an answer. It uses the Call Automation Play API with support for text to speech to play back the answer provided by OpenAI to the caller.

Prerequisites

Before you run the sample for the first time

Open the application.yml file in the resources folder to configure the following settings:

  • connectionstring: Azure Communication Services resource connection string.
  • basecallbackuri: Base URL of the app. For local development, use the dev tunnel URL.
  • cognitiveServicesUrl: Azure AI services endpoint.
  • azureOpenAiServiceKey: Azure OpenAI service key.
  • azureOpenAiServiceEndpoint: Azure OpenAI service endpoint.
  • openAiModelName: Azure OpenAI model name.

Set up and host your Azure dev tunnel

With dev tunnels, you can share local web services that are hosted on the internet. Use the following commands to connect your local development environment to the public internet. This process creates a tunnel with a persistent endpoint URL and allows anonymous access. We use this endpoint to notify your application of calling events from Call Automation.

devtunnel create --allow-anonymous
devtunnel port create -p 8080
devtunnel host

Run the application

  • Go to the directory that contains the pom.xml file and use the following mvn commands:

    • Compile the application: mvn compile
    • Build the package: mvn package
    • Execute the app: mvn exec:java

  • Access the Swagger UI at http://localhost:8080/swagger-ui.html.

    • Try the GET /outboundCall to run the sample application.

Now you have a running application. The best way to test this sample is to place a call to your Azure Communication Services phone number and talk to your intelligent agent.

Related content

Download code

Find the project for this sample on GitHub. You can download this code and run it locally to try it for yourself.

Overview

This sample is a server-side application that helps you create a virtual assistant that can handle calls by using Call Automation. The virtual assistant responds to customers by using the newly announced integration with Azure AI services to provide AI capabilities such as text to speech and speech to text, along with smart responses provided by Azure OpenAI.

This Azure Communication Services Call Automation AI sample demonstrates how to use the Call Automation SDK to answer an inbound call. It recognizes user voice input by using Call Automation to recognize the API with support for speech to text. When the system recognizes the input, it sends that information to OpenAI for an answer. It uses the Call Automation Play API with support for text to speech to play back the answer provided by OpenAI to the caller.

Prerequisites

Setup instructions

Before you run this sample, you need to set up the resources mentioned in the preceding section with the following configuration updates.

Step 1: Set up and host your Azure dev tunnel

With dev tunnels, you can share local web services that are hosted on the internet. Use the following commands to connect your local development environment to the public internet. This process creates a tunnel with a persistent endpoint URL and allows anonymous access. We use this endpoint to notify your application of calling events from Call Automation.

devtunnel create --allow-anonymous
devtunnel port create -p 8080
devtunnel host

Step 2: Add a managed identity to the Azure Communication Services resource that connects to the Azure AI services resource

Follow the instructions in Connect Azure Communication Services with Azure AI services.

Step 3: Add the required API keys and endpoints

Open the .env file to configure the following settings:

  • CONNECTION_STRING: Azure Communication Services resource connection string.
  • CALLBACK_URI: Base URL of the app. For local development, replace the dev tunnel URL.
  • COGNITIVE_SERVICE_ENDPOINT: Azure AI service endpoint.
  • AZURE_OPENAI_SERVICE_KEY: Azure OpenAI service key.
  • AZURE_OPENAI_SERVICE_ENDPOINT: Azure OpenAI endpoint.
  • AZURE_OPENAI_DEPLOYMENT_MODEL_NAME: Azure OpenAI deployment name.
  • AGENT_PHONE_NUMBER: Agent phone number to transfer the call to resolve queries.

Run the application

  1. Open a new PowerShell window, use the cd command to open the callautomation-openai-sample folder, and run npm run dev.
  2. The browser opens with a page. If it doesn't, go to http://localhost:8080/.
  3. Register an Azure Event Grid webhook for the IncomingCall event that points to your DevTunnel URI. For more information, see Incoming call concepts.

Now you have a running application. The best way to test this sample is to place a call to your Azure Communication Services phone number and talk to your intelligent agent.

Related content

Download code

Find the project for this sample on GitHub. You can download this code and run it locally to try it for yourself.

Overview

This sample is a server-side application that helps you create a virtual assistant that can handle calls by using Call Automation. It also enables your assistant to respond to customers by using Azure AI services, which provide AI capabilities such as text to speech and speech to text, along with smart responses provided by Azure OpenAI.

This Azure Communication Services Call Automation AI sample demonstrates how to use the Call Automation SDK to answer an inbound call. It recognizes user voice input by using Call Automation to recognize the API with support for speech to text. When the system recognizes the input, it sends the information to OpenAI for an answer. It uses the Call Automation Play API with support for text to speech to play back the answer provided by OpenAI to the caller.

Prerequisites

Setup instructions

Before you run this sample, you need to set up the resources mentioned in the preceding section with the following configuration updates.

Step 1: Set up a Python environment

Create and activate a Python virtual environment and install the required packages by using the following command:

pip install -r requirements.txt

Step 2: Set up and host your Azure dev tunnel

With dev tunnels, you can share local web services that are hosted on the internet. Use the following commands to connect your local development environment to the public internet. This process creates a tunnel with a persistent endpoint URL and allows anonymous access. We use this endpoint to notify your application of calling events from Call Automation.

devtunnel create --allow-anonymous
devtunnel port create -p 8080
devtunnel host

Step 3: Add a managed identity to the Azure Communication Services resource that connects to the Azure AI services resource

Follow the instructions in Connect Azure Communication Services with Azure AI services.

Step 4: Add the required API keys and endpoints

Open the main.py file to configure the following settings:

  • CALLBACK_URI_HOST: Your dev tunnel endpoint.
  • COGNITIVE_SERVICE_ENDPOINT: The Azure AI services endpoint.
  • ACS_CONNECTION_STRING: Azure Communication Services resource connection string.
  • AZURE_OPENAI_SERVICE_KEY: Azure OpenAI service key.
  • AZURE_OPENAI_SERVICE_ENDPOINT: Azure OpenAI service endpoint.
  • AZURE_OPENAI_DEPLOYMENT_MODEL_NAME: Azure OpenAI model name.
  • AGENT_PHONE_NUMBER: Agent phone number to transfer call.

Run the application

  1. Go to the callautomation-openai-sample folder and run main.py in debug mode. You can also use the command python ./main.py to run it from PowerShell, a command prompt, or a Unix terminal.
  2. The browser opens with a page. If it doesn't, go to http://localhost:8080/ or your dev tunnel URL.
  3. Register an Azure Event Grid webhook for the IncomingCall event that points to your DevTunnel URI. For more information, see Incoming call concepts.

Now you have a running application. The best way to test this sample is to place a call to your Azure Communication Services phone number and talk to your intelligent agent.

Related content

  • Last updated on