Extract and analyze call center data

Azure Blob Storage

Azure Speech in Foundry Tools

Foundry Tools

Power BI

This article describes a batch processing architecture that extracts insights from customer conversations in a call center. The solution uses Foundry Tools and Azure OpenAI to analyze post-call transcripts after calls complete rather than in near real time. With this approach, you can analyze call intent and sentiment, extract key entities, and summarize calls offline to help improve customer interactions and satisfaction.

Architecture

The architecture diagram shows the post-call analytics workflow. In step 1, a caller connects to a call-center agent through a person-to-person conversation line, and both connect to a telephony server. An arrow labeled file upload points from the telephony server to Azure Blob Storage to store recorded audio files. In step 2, Azure Speech batch transcription reads the audio files from Blob Storage and writes the transcription results back into storage. In step 3, language enrichment processes the transcripts. In step 4, the Azure App Service app sends the enriched transcripts to Azure OpenAI for analysis. In step 5, the processed analytics data flows from Blob Storage to three destinations in the interact and visualize section: Power BI for insights visualization, a web app, and a customer relationship management (CRM) system that displays call summaries, call reasons, and detailed call history.

Download a PowerPoint file of this architecture.

Data flow

The following data flow corresponds to the previous diagram:

1. The telephony server records a phone call between an agent and a customer and stores it in Azure Blob Storage. Audio files are uploaded to an Azure Storage account via a supported method, such as the UI-based tool Azure Storage Explorer or a Storage SDK or API.

2. An Azure function uses one of the following triggers to start the intelligent transcription process:

   • Timer trigger: Configure a time-based trigger to process a batch of audio files accumulated over a specified time period.

   • Blob trigger: Configure a blob trigger to initiate intelligent transcription when an audio file is uploaded to the blob container.

3. The Azure function triggers a custom application that runs the following steps in sequence. It calls Azure Speech batch transcription to transcribe the audio files and optionally saves the raw transcription file in Blob Storage for future reference. The app passes the raw data to Azure Language to detect and redact personal data in the transcript.

4. The web app sends the redacted data to Azure OpenAI text processing models such as GPT-5 models to perform various post-call analytics, including identifying call intent and sentiment, extracting entities, or summarizing the conversation to evaluate the call's effectiveness. The solution stores the processed output in Azure Storage for visualization or consumption by downstream applications or other datastores used for reporting.

5. Power BI visualizes the post-call analytics based on the criteria that the business defines. You can also store this output in a customer relationship management (CRM) system, so agents have contextual information about why the customer called and can quickly solve potential problems. This automated process saves agents time and effort.

Components

• Blob Storage is an object storage service that supports libraries for languages such as .NET, Node.js, and Python. Applications can access files on Blob Storage by using HTTP or HTTPS. Blob Storage has hot, cool, and archive access tiers to store large amounts of data and optimize cost. In this architecture, Blob Storage stores raw audio files and processed outputs.

• Foundry Models is the model catalog and model access layer in Microsoft Foundry. It provides access to multiple models that have different capabilities, including language models, audio models, image and video generation models, and embeddings models. You can access the catalog through REST APIs, SDKs, or the Foundry portal. In this architecture, Foundry Models provides AI capabilities for transcription and analysis.

• Speech is an AI-based API that includes speech capabilities such as speech to text, text to speech, speech translation, and speaker recognition. In this architecture, Speech batch transcription converts recorded audio files into text.

• Language is a service that consolidates the Azure natural language processing services into a unified API. In this architecture, Language detects and redacts personal data from call transcripts.

• Language Studio is a UI-based tool for exploring, building, tagging, training, and deploying custom language models. In this architecture, Language Studio customizes language processing features for your specific call-center domain.

• Power BI is a software as a service (SaaS) that provides visual and interactive insights for business analytics. The service includes transformation capabilities and connects to other data sources. In this architecture, Power BI visualizes post-call analytics based on business requirements.

Alternatives

You can choose the following workflows, depending on your scenario.

• Detect personal data by configuring the guardrails in Foundry and applying those guardrails to deployed models, including large language models (LLMs) and Azure OpenAI models. You can filter different types of personal data, such as email, phone number, address, financial information, and government ID. The system supports two modes:

  • Annotate, which flags the personal data in the output

  • Annotate and block, which blocks the entire output if the system detects personal data

  Set these modes for each personal category individually.

• Use the Fast Transcription API to convert speech to text synchronously. LLM speech (preview) uses an LLM-enhanced speech model to transcribe audio files and includes built-in features like translation.

• Use the speech analytics feature in Azure Content Understanding to orchestrate the batch post-call analytics process.

• Use GPT audio speech-to-text models to generate audio transcripts and store them in Blob Storage for call analytics.

• Use the ingestion client to deploy the post-call analytics solution to Azure. This solution uses Speech and Language services as the intelligence layer, without the generative AI capabilities that Azure OpenAI models provide.

• For virtual agents, use:

  • The Voice Live API for speech-to-speech conversations through telephony integration without a public switched telephone network (PSTN). The Voice Live API supports different generative AI models, including Azure OpenAI realtime models. If you choose a nonmultimodal model such as GPT-4o, Azure speech to text automatically becomes the audio input. You can store the audio and transcription of the conversation in Blob Storage to analyze and gather insights for your business. The Voice Live API doesn't support session initiation protocol (SIP), but it works with external SIP trunking solutions.

  • Use GPT-realtime models to achieve low-latency speech-to-speech conversations. You can also use the GPT Realtime API via WebRTC, WebSockets, or SIP to send audio input and receive audio responses in real time and store them with the transcription for analytics.

Scenario details

This solution uses the Batch Transcription API in Speech to convert call-center audio into written text. Language redacts sensitive information in the conversation transcription. Azure OpenAI extracts insights from customer conversations to improve call-center efficiency and customer satisfaction.

Use this solution to process transcribed text, recognize and remove sensitive information, and extract insights like call reason, resolution status, sentiment, and product or service trends based on query volume or customer complaints. Scale the services and pipeline to accommodate any volume of recorded data.

Potential use cases

This solution benefits organizations across multiple industries that have customer support agents. Post-call analytics can help improve the company's products, services, and customer support systems. The solution applies to organizations that record conversations, including organizations that have customer-facing agents, internal call centers, or support desks.

Considerations

These considerations implement the pillars of the Azure Well-Architected Framework, which is a set of guiding tenets that you can use to improve the quality of a workload. For more information, see Well-Architected Framework.

Reliability

Reliability helps ensure that your application can meet the commitments that you make to your customers. For more information, see Design review checklist for Reliability.

• Find the availability service-level agreement (SLA) for each component in SLAs for online services.

• Design high-availability applications with Storage accounts by reviewing the configuration options.

• Ensure resiliency of the compute services and data stores in this scenario by testing failure modes for core services such as Azure Functions and Storage. For more information, see Reliability guides by service.

Security

Security provides assurances against deliberate attacks and the misuse of your valuable data and systems. For more information, see Design review checklist for Security.

• Implement data protection, identity and access management, and network security recommendations for Blob Storage, Foundry Tools, and Azure OpenAI.

• Configure Foundry Tools virtual networks.

Cost Optimization

Cost Optimization focuses on ways to reduce unnecessary expenses and improve operational efficiencies. For more information, see Design review checklist for Cost Optimization.

The total cost of this solution depends on the pricing tier of your services. Factors that can affect the price of each component are:

• The number of documents that you process.
• The number of concurrent requests that your application receives.
• The size of the data that you store after processing.
• Your deployment region.

For more information, see the following resources:

• Foundry
• Azure OpenAI pricing
• Blob Storage pricing
• Language pricing

Use the Azure pricing calculator to estimate your solution cost.

Performance Efficiency

Performance Efficiency refers to your workload's ability to scale to meet user demands efficiently. For more information, see Design review checklist for Performance Efficiency.

When you process high volumes of data, the system can expose performance bottlenecks. To ensure proper performance efficiency, learn about scaling options to use with the Foundry Tools autoscale feature.

The Batch Speech API handles high volumes, but other Foundry Tools APIs might have request limits, depending on the subscription tier. Consider containerizing Foundry Tools APIs to avoid slowdowns during large-volume processing. Containers provide deployment flexibility in the cloud and on-premises. Use containers to mitigate side effects of new version rollouts. For more information, see Container support in Foundry Tools.

Contributors

Microsoft maintains this article. The following contributors wrote this article.

Principal authors:

• Dixit Arora | Senior Customer Engineer, EngOps CRE
• Jyotsna Ravi | Sr. Account Executive

To see nonpublic LinkedIn profiles, sign in to LinkedIn.

Next steps

• Speech overview
• Foundry Models
• Introduction to Blob Storage
• Language overview
• Introduction to Azure Data Lake Storage
• Power BI overview
• Post-call transcription and analytics
• Create custom language and acoustic models

This article describes a batch processing architecture that extracts insights from customer conversations in a call center. The solution uses Foundry Tools and Azure OpenAI to analyze post-call transcripts after calls complete rather than in near real time. With this approach, you can analyze call intent and sentiment, extract key entities, and summarize calls offline to help improve customer interactions and satisfaction.

Architecture

The architecture diagram shows the post-call analytics workflow. In step 1, a caller connects to a call-center agent through a person-to-person conversation line, and both connect to a telephony server. An arrow labeled file upload points from the telephony server to Azure Blob Storage to store recorded audio files. In step 2, Azure Speech batch transcription reads the audio files from Blob Storage and writes the transcription results back into storage. In step 3, language enrichment processes the transcripts. In step 4, the Azure App Service app sends the enriched transcripts to Azure OpenAI for analysis. In step 5, the processed analytics data flows from Blob Storage to three destinations in the interact and visualize section: Power BI for insights visualization, a web app, and a customer relationship management (CRM) system that displays call summaries, call reasons, and detailed call history.

Download a PowerPoint file of this architecture.

Data flow

The following data flow corresponds to the previous diagram:

1. The telephony server records a phone call between an agent and a customer and stores it in Azure Blob Storage. Audio files are uploaded to an Azure Storage account via a supported method, such as the UI-based tool Azure Storage Explorer or a Storage SDK or API.

2. An Azure function uses one of the following triggers to start the intelligent transcription process:

   • Timer trigger: Configure a time-based trigger to process a batch of audio files accumulated over a specified time period.

   • Blob trigger: Configure a blob trigger to initiate intelligent transcription when an audio file is uploaded to the blob container.

3. The Azure function triggers a custom application that runs the following steps in sequence. It calls Azure Speech batch transcription to transcribe the audio files and optionally saves the raw transcription file in Blob Storage for future reference. The app passes the raw data to Azure Language to detect and redact personal data in the transcript.

4. The web app sends the redacted data to Azure OpenAI text processing models such as GPT-5 models to perform various post-call analytics, including identifying call intent and sentiment, extracting entities, or summarizing the conversation to evaluate the call's effectiveness. The solution stores the processed output in Azure Storage for visualization or consumption by downstream applications or other datastores used for reporting.

5. Power BI visualizes the post-call analytics based on the criteria that the business defines. You can also store this output in a customer relationship management (CRM) system, so agents have contextual information about why the customer called and can quickly solve potential problems. This automated process saves agents time and effort.

Components

• Blob Storage is an object storage service that supports libraries for languages such as .NET, Node.js, and Python. Applications can access files on Blob Storage by using HTTP or HTTPS. Blob Storage has hot, cool, and archive access tiers to store large amounts of data and optimize cost. In this architecture, Blob Storage stores raw audio files and processed outputs.

• Foundry Models is the model catalog and model access layer in Microsoft Foundry. It provides access to multiple models that have different capabilities, including language models, audio models, image and video generation models, and embeddings models. You can access the catalog through REST APIs, SDKs, or the Foundry portal. In this architecture, Foundry Models provides AI capabilities for transcription and analysis.

• Speech is an AI-based API that includes speech capabilities such as speech to text, text to speech, speech translation, and speaker recognition. In this architecture, Speech batch transcription converts recorded audio files into text.

• Language is a service that consolidates the Azure natural language processing services into a unified API. In this architecture, Language detects and redacts personal data from call transcripts.

• Language Studio is a UI-based tool for exploring, building, tagging, training, and deploying custom language models. In this architecture, Language Studio customizes language processing features for your specific call-center domain.

• Power BI is a software as a service (SaaS) that provides visual and interactive insights for business analytics. The service includes transformation capabilities and connects to other data sources. In this architecture, Power BI visualizes post-call analytics based on business requirements.

Alternatives

You can choose the following workflows, depending on your scenario.

• Detect personal data by configuring the guardrails in Foundry and applying those guardrails to deployed models, including large language models (LLMs) and Azure OpenAI models. You can filter different types of personal data, such as email, phone number, address, financial information, and government ID. The system supports two modes:

  • Annotate, which flags the personal data in the output

  • Annotate and block, which blocks the entire output if the system detects personal data

  Set these modes for each personal category individually.

• Use the Fast Transcription API to convert speech to text synchronously. LLM speech (preview) uses an LLM-enhanced speech model to transcribe audio files and includes built-in features like translation.

• Use the speech analytics feature in Azure Content Understanding to orchestrate the batch post-call analytics process.

• Use GPT audio speech-to-text models to generate audio transcripts and store them in Blob Storage for call analytics.

• Use the ingestion client to deploy the post-call analytics solution to Azure. This solution uses Speech and Language services as the intelligence layer, without the generative AI capabilities that Azure OpenAI models provide.

• For virtual agents, use:

  • The Voice Live API for speech-to-speech conversations through telephony integration without a public switched telephone network (PSTN). The Voice Live API supports different generative AI models, including Azure OpenAI realtime models. If you choose a nonmultimodal model such as GPT-4o, Azure speech to text automatically becomes the audio input. You can store the audio and transcription of the conversation in Blob Storage to analyze and gather insights for your business. The Voice Live API doesn't support session initiation protocol (SIP), but it works with external SIP trunking solutions.

  • Use GPT-realtime models to achieve low-latency speech-to-speech conversations. You can also use the GPT Realtime API via WebRTC, WebSockets, or SIP to send audio input and receive audio responses in real time and store them with the transcription for analytics.

Scenario details

This solution uses the Batch Transcription API in Speech to convert call-center audio into written text. Language redacts sensitive information in the conversation transcription. Azure OpenAI extracts insights from customer conversations to improve call-center efficiency and customer satisfaction.

Use this solution to process transcribed text, recognize and remove sensitive information, and extract insights like call reason, resolution status, sentiment, and product or service trends based on query volume or customer complaints. Scale the services and pipeline to accommodate any volume of recorded data.

Potential use cases

This solution benefits organizations across multiple industries that have customer support agents. Post-call analytics can help improve the company's products, services, and customer support systems. The solution applies to organizations that record conversations, including organizations that have customer-facing agents, internal call centers, or support desks.

Considerations

These considerations implement the pillars of the Azure Well-Architected Framework, which is a set of guiding tenets that you can use to improve the quality of a workload. For more information, see Well-Architected Framework.

Reliability

Reliability helps ensure that your application can meet the commitments that you make to your customers. For more information, see Design review checklist for Reliability.

• Find the availability service-level agreement (SLA) for each component in SLAs for online services.

• Design high-availability applications with Storage accounts by reviewing the configuration options.

• Ensure resiliency of the compute services and data stores in this scenario by testing failure modes for core services such as Azure Functions and Storage. For more information, see Reliability guides by service.

Security

Security provides assurances against deliberate attacks and the misuse of your valuable data and systems. For more information, see Design review checklist for Security.

• Implement data protection, identity and access management, and network security recommendations for Blob Storage, Foundry Tools, and Azure OpenAI.

• Configure Foundry Tools virtual networks.

Cost Optimization

Cost Optimization focuses on ways to reduce unnecessary expenses and improve operational efficiencies. For more information, see Design review checklist for Cost Optimization.

The total cost of this solution depends on the pricing tier of your services. Factors that can affect the price of each component are:

• The number of documents that you process.
• The number of concurrent requests that your application receives.
• The size of the data that you store after processing.
• Your deployment region.

For more information, see the following resources:

• Foundry
• Azure OpenAI pricing
• Blob Storage pricing
• Language pricing

Use the Azure pricing calculator to estimate your solution cost.

Performance Efficiency

Performance Efficiency refers to your workload's ability to scale to meet user demands efficiently. For more information, see Design review checklist for Performance Efficiency.

When you process high volumes of data, the system can expose performance bottlenecks. To ensure proper performance efficiency, learn about scaling options to use with the Foundry Tools autoscale feature.

The Batch Speech API handles high volumes, but other Foundry Tools APIs might have request limits, depending on the subscription tier. Consider containerizing Foundry Tools APIs to avoid slowdowns during large-volume processing. Containers provide deployment flexibility in the cloud and on-premises. Use containers to mitigate side effects of new version rollouts. For more information, see Container support in Foundry Tools.

Contributors

Microsoft maintains this article. The following contributors wrote this article.

Principal authors:

• Dixit Arora | Senior Customer Engineer, EngOps CRE
• Jyotsna Ravi | Sr. Account Executive

To see nonpublic LinkedIn profiles, sign in to LinkedIn.

Next steps

• Speech overview
• Foundry Models
• Introduction to Blob Storage
• Language overview
• Introduction to Azure Data Lake Storage
• Power BI overview
• Post-call transcription and analytics
• Create custom language and acoustic models