Quick Start: Integrating Azure OpenAI with ACS Job Router

Integrate ACS Job Router with Azure OpenAI. Use Azure OpenAI to pair your jobs to agents.

Prerequisites

• Create an Azure OpenAI resource. Setup Guide
• Create an Azure Communication Services resource. Setup Guide
• Clone the GitHub solution. Integrating Azure OpenAI with ACS Job Router
• Visual Studio Code Installed. Visual Studio Code
• Azure Functions Extension for Visual Studio Code. Azure Function Extension for Visual Studio Code

Overview

This quick start demonstrates how to integrate Azure OpenAI with ACS Job Router to intelligently choose the best-suited worker based on performance indicators for an incoming job.

Console application

• Manages ACS resources including policies and queues.
• Simulates job queuing and allocation.

Azure Function project

• Hosts API for Azure OpenAI integration.
• Manages worker scoring and job labels.

This guide covers two main projects:

• A .NET console application to interact with ACS Job Router.
• An Azure Function project for OpenAI integration with ACS Job Router.

The console application is set up to provision the following ACS resources:

• A distribution policy that lets ACS Job Router understand how to generate offers for workers. This application is configured to provision a Best-Worker mode distribution policy that uses a Function Router Rule for scoring workers.

• A queue with the best-worker mode distribution policy attached.

• Five workers that are registered to a queue with three chosen performance indicator values populated as labels.

• Creates a Job in ACS Job Router and lets the user know which worker Azure OpenAI scored the highest based on the performance indicator labels.

  

The Azure Function project is set up to interact with a deployed Azure OpenAI model:

• The Azure function receives a request from ACS Job Router with a payload containing the worker's labels (performance indicators).
• The Azure function extracts those values from the request.
• The Azure function was preconfigured with a prompt to explain to the Azure OpenAI model how to interpret each one of these performance indicators and to ask Azure OpenAI to score each worker based on these values to find the most suitable agent for a new job.

  Note

  The prompt can be updated for any desired outcome. For example, you could modify the prompt to weigh the Average Handling Time as the most important data point or ask the model to optimize for customer satisfaction.

• The Azure Function sends a request with the configured prompts and worker performance indicators, and Azure OpenAI responds back with a JSON Object containing the scores generated by Azure OpenAI.
• The Azure Function then sends these scores back to ACS Job Router.
• ACS Job Router then sends an offer to the worker that was scored the highest by Azure OpenAI.

Performance indicators used in this project

In this guide, we've chosen three performance indicators based on typical contact center performance data points. Workers are evaluated based on:

• CSAT: Customer satisfaction.
• Outcome: Issue resolution rate.
• AHT: Average handling time.

Understanding ACS Job Router

• Learn about ACS Job Router. Job Router Concepts
• Configure the BestWorker Distribution Policy using an Azure Function Scoring Rule. Customize Worker Scoring

Understanding Azure Function deployments

• Learn about Azure Function Deployments. Azure Function Deployments

Understanding Azure OpenAI prompts

• Learn about Azure OpenAI prompt Engineering Techniques. Prompt Engineering Techniques

Deployment and execution

1. Open the OpenAiScoringFunction project in Visual Studio Code with the Azure Function Extension installed. Select 'Create Function App in Azure...'

   

2. After selecting your Subscription, enter a unique name for your function app.

   

3. Once your Function App is created, right-click on your App and select 'Deploy Function App...'

4. Open the Azure portal and go to your Azure OpenAI resource, then go to the Azure AI Foundry portal. From here, navigate to the Deployments tab and select "+ Create new deployment"

   1. Select a model that can perform completions

      Azure OpenAI Service models

   2. b. Give your model a Deployment name and select “Create”

      

5. Once your Azure OpenAI Model is created, copy down the 'Endpoint', 'Keys', and 'Region'

   

6. In the Azure portal, navigate to your newly created Function App Environmental Variables blade and create the following variables:

   

Name	Value	Description
OpenAIBaseURI	{Endpoint}	Endpoint URI from OpenAI Resource
OpenAIAPIKey	{Key}	Key from OpenAI Resource
DeploymentName	{DeploymentName}	Deployment Name from OpenAI Resource
Preprompt	You're helping pair a customer with an agent in a contact center. You'll evaluate the best available agent based on their performance indicators below. CSAT holds the average customer satisfaction score between 1 and 3, higher is better. Outcome is a score between 0 and 1, higher is better. AHT is average handling time, lower is better. If AHT provided is 00:00, please ignore it in the scoring.	Prompt containing preprocessing instructions for the Azure OpenAI model
Postprompt	Respond with only a json object with agent ID as the key, and scores based on suitability for this customer as the value in a range of 0 to 1. Don't include any other information.	Prompt containing postprocessing instructions for the Azure OpenAI model
DefaultCSAT	1.5	Default CSAT score for workers missing this label
DefaultOutcome	0.5	Default Outcome score for workers missing this label
DefaultAHT	10:00	Default AHT for workers missing this label

7. Go to the Overview blade of your function app.

   1. Select the newly created function.

      

   2. Select the "Get Function URL" button and copy down the URL.

      

8. Navigate to your Azure Communication Services resource, click on the "Keys" blade and copy down your Connection string.

9. Open the JR_AOAI_Integration Console application and open the appsettings.json file to update the following config settings.

   Note

   The "AzureFunctionUri" will be the everything in the function url before the "?code=" and the "AzureFunctionKey" will everything after the the "?code=" in the function url.

   

10. Run the application and follow the on-screen instructions to Create a Job.

    • Once a job has been created the console application will let you know who scored the highest and has received the offer. To see the prompts sent to your OpenAI model and scores given to your workers and sent back to Job Router. Go to your Function and select the Monitor Tab and watch the logs as you are creating a job in the console application.

Experimentation

Various experiments can be conducted within this project, such as:

• Experimenting with the PrePrompt string (in the Environment Variables of the Azure Function) to further tune scores provided by Azure OpenAI.
• Add other performance indicator labels to workers, updating the OpenAiScorer class in the OpenAIScorerFunction project to account for new labels and updating the prompts, default performance indicators in the Environment Variables of your function, and add the new performance indicators into the appSettings.json file under each worker.
• Implement logic to update the values of the performance indicator labels as jobs are completed by each worker. This could be by adding persistence or a cache to your application to store these values. Labels are routable attributes in Job Router. If a worker's labels are updated, any offers issued for that worker will be revoked. Consider updating labels at a point when the worker isn't expecting offers (AvailableForOffers – false, or when the worker’s capacity is consumed).