Edit

Share via

Call Automation and Azure OpenAI

This sample application was demonstrated during Microsoft Ignite 2024. It highlights the integration of Azure Communication Services with Azure OpenAI to enable intelligent conversational agents.

This article covers what this sample does and what you need as prerequisites before the sample runs locally on your machine.

Download code

Find the project for this sample on GitHub. You can download this code and run it locally to try it for yourself.

Overview

This server-side application helps you create a virtual assistant that can handle phone calls and respond to customers by using Azure OpenAI. The virtual assistant quickly provides real-time answers.

Prerequisites

Note

Bidirectional streaming in Azure Communication Services is generally available, but the Azure OpenAI real-time API has its own release schedule. For up-to-date information on the API's availability, see Azure OpenAI in Azure AI Foundry models.

Setup instructions

Before you run this sample, you need to set up the resources with the following configuration updates.

Step 1: Set up and host your dev tunnel

With dev tunnels, you can share local web services that are hosted on the internet. Use the commands to connect your local development environment to the public internet. This process creates a tunnel with a persistent endpoint URL and allows anonymous access. We use this endpoint to notify your application of calling events from Call Automation.

devtunnel create --allow-anonymous
devtunnel port create -p 5165
devtunnel host

Step 2: Add the required API keys and endpoints

Open the appsettings.json file to configure the following settings:

    - DevTunnelUri: your dev tunnel endpoint
    - AcsConnectionString: Azure Communication Service resource's connection string.
    - AzureOpenAIServiceKey: Open AI's Service Key
    - AzureOpenAIServiceEndpoint: Open AI's Service Endpoint
    - AzureOpenAIDeploymentModelName: Open AI's Model name

Run the application

  1. Ensure that your AzureDevTunnel URI is active and points to the correct port of your local host application.
  2. Run dotnet run to build and run the sample application.
  3. Register an Azure Event Grid webhook for the IncomingCall event that points to your DevTunnel URI. For more information, see Incoming call concepts. For example:
 https://<devtunelurl>/api/incomingCall

Now you have a running application. The best way to test this sample is to place a call to your Azure Communication Services phone number and talk to your intelligent agent.

Related content

Download code

Find the project for this sample on GitHub. You can download this code and run it locally to try it for yourself.

Overview

This server-side application helps you create a virtual assistant that can handle phone calls and respond to customers by using Azure OpenAI. The virtual assistant quickly provides real-time answers.

Prerequisites

Note

Bidirectional streaming in Azure Communication Services is generally available, but the Azure OpenAI real-time API has its own release schedule. For up-to-date information on the API's availability, see Azure OpenAI in Azure AI Foundry models.

Before you run the sample for the first time

  1. Open an instance of PowerShell, a Windows terminal, a command prompt, or the equivalent, and go to the directory where you want to clone the sample.
  2. Use the command git clone https://github.com/Azure-Samples/communication-services-javascript-quickstarts.git.
  3. Use the command cd to access the callautomation-az-openai-voice folder.
  4. From the root of the folder, and with a node installed, run npm install.

Step 1: Set up and host your dev tunnel

With dev tunnels, you can share local web services that are hosted on the internet. Use the commands to connect your local development environment to the public internet. This process creates a tunnel with a persistent endpoint URL and allows anonymous access. We use this endpoint to notify your application of calling events from Call Automation.

devtunnel create --allow-anonymous
devtunnel port create -p 8080
devtunnel host

Step 2: Add the required API keys and endpoints

Open the .env file to configure the following settings:

  • CONNECTION_STRING: Azure Communication Services resource connection string.
  • CALLBACK_URI: Base URL of the app. For local developments, replace the dev tunnel URL.
  • AZURE_OPENAI_SERVICE_KEY: Azure OpenAI service key.
  • AZURE_OPENAI_SERVICE_ENDPOINT: Azure OpenAI endpoint.
  • AZURE_OPENAI_DEPLOYMENT_MODEL_NAME: Azure OpenAI deployment name.

Run the application

Run the app locally

  1. Open a new PowerShell window, use the cd command to open the callautomation-az-openai-voice folder, and run npm run dev.

  2. The browser opens with a page. If it doesn't, go to http://localhost:8080/.

  3. Register an Azure Event Grid webhook for the IncomingCall event that points to your 8080 port URI. For more information, see Incoming call concepts. For example:

     https://<devtunelurl>/api/incomingCall

Now you have a running application. The best way to test this sample is to place a call to your Azure Communication Services phone number and talk to your intelligent agent.

Related content

  • Last updated on