Integrate Azure Communication calling with Realtime gpt model

Question

Currently, the project utilizes Twilio to make outbound calls, integrating Azure's real-time GPT models to enable interactive, real-time conversations.
The goal is to switch from Twilio to the Azure Call Automation Service for handling outbound calls.
The implementation is planned in either Java or Python.
Is it feasible to achieve the same real-time conversational experience using Azure Call Automation Service?

Answer 1

Hi @Ashley
Yes, you can create the same real-time conversational experience using Azure Communication Services (ACS) Call Automation with Azure’s real-time GPT or voice models instead of Twilio. Microsoft’s documentation and samples show that ACS Call Automation can manage outbound PSTN calls, process voice audio, send recognized text to an Azure OpenAI model for intelligent responses, and play back generated audio with text-to-speech, enabling an AI-powered voice agent. The official Call Automation AI sample demonstrates how the SDK handles call events and speech input, sends recognized text to Azure OpenAI, and returns the response via TTS.

Azure documentation also details how to use bidirectional audio streaming with ACS, allowing you to send and receive live audio for integration with real-time voice processing services like the Azure OpenAI Realtime API or Voice Live API for speech-to-text, generative AI, and TTS.

By combining ACS Call Automation for outbound calls with Azure OpenAI real-time or Voice Live API, you can build a real-time, interactive voice solution in Java or Python that replaces Twilio and GPT, following Microsoft’s official guidance.
https://learn.microsoft.com/en-us/azure/communication-services/concepts/call-automation/audio-streaming-concept?utm_source=chatgpt.com
https://learn.microsoft.com/en-us/azure/communication-services/samples/call-automation-ai?utm_source=chatgpt.com&pivots=programming-language-javascript
Let me know if you have any further assisstences needed.