Quickstart: Getting started with the Personalizer client library

Imagine a scenario where a grocery e-retailer wishes to increase revenue by showing relevant and personalized products to each customer visiting their website. On the main page, there's a "Featured Product" section that displays a prepared meal product to prospective customers. However, the e-retailer would like to determine how to show the right product to the right customer in order to maximize the likelihood of a purchase.

In this quick-start, you'll learn how to use the Azure Personalizer service to do solve this problem in an automated, scalable, and adaptable fashion using the power of reinforcement learning. You'll learn how to create actions and their features, context features, and reward scores. You'll use the Personalizer client library to make calls to the Rank and Reward APIs. You'll also run a cycle of Rank and Reward calls for three example users.

You will need to install the Personalizer client library for .NET to:

  • Authenticate the quickstart example client with a Personalizer resource in Azure.
  • Send context and action features to the Reward API, which will return the best action from the Personalizer model
  • Send a reward score to the Rank API and train the Personalizer model.

Reference documentation | Library source code | Package (NuGet) | .NET code sample

Prerequisites

  • Azure subscription - Create one for free
  • The current version of .NET Core.
  • Once you have your Azure subscription, create a Personalizer resource in the Azure portal to get your key and endpoint. After it deploys, click Go to resource.
    • You will need the key and endpoint from the resource you create to connect your application to the Personalizer API. You'll paste your key and endpoint into the code below later in the quickstart.
    • You can use the free pricing tier (F0) to try the service, and upgrade later to a paid tier for production.

Setting Up

Change the model update frequency

In the Azure portal, in the Personalizer resource on the Configuration page, change the Model update frequency to 30 seconds. This short duration will train the service rapidly, allowing you to see how the top action changes for each iteration.

Change model update frequency

When a Personalizer loop is first instantiated, there is no model since there has been no Reward API calls to train from. Rank calls will return equal probabilities for each item. Your application should still always rank content using the output of RewardActionId.

Change the reward wait time

In the Azure portal, in the Personalizer resource on the Configuration page, change the Reward wait time to 10 minutes. This short duration will train the service rapidly, allowing you to see how the top action changes for each iteration.

Change reward wait time

Create a new C# application

Create a new .NET Core application in your preferred editor or IDE.

In a console window (such as cmd, PowerShell, or Bash), use the dotnet new command to create a new console app with the name personalizer-quickstart. This command creates a simple "Hello World" C# project with a single source file: Program.cs.

dotnet new console -n personalizer-quickstart

Change your directory to the newly created app folder. You can build the application with:

dotnet build

The build output should contain no warnings or errors.

...
Build succeeded.
 0 Warning(s)
 0 Error(s)
...

Install the client library

Within the application directory, install the Personalizer client library for .NET with the following command:

dotnet add package Microsoft.Azure.CognitiveServices.Personalizer --version 1.0.0

Tip

If you're using the Visual Studio IDE, the client library is available as a downloadable NuGet package.

From the project directory, open the Program.cs file in your preferred editor or IDE. Add the following using directives:

using Microsoft.Azure.CognitiveServices.Personalizer;
using Microsoft.Azure.CognitiveServices.Personalizer.Models;

Object model

The client is a PersonalizerClient object that authenticates to Azure using Microsoft.Rest.ServiceClientCredentials containing your key.

To request the best action from Personalizer, create a RankRequest containing a list of RankableActions that Personalizer will choose from, and the context features. The RankRequest will be passed to the client.Rank method, which returns a RankResponse.

To send a reward score to Personalizer, create a RewardRequest with the event ID corresponding to the Rank call that returned the best action, and the reward score. Then, pass it to the client.Reward method.

Later in this quick-start, we'll define an example reward score. However, the reward is one of the most important considerations when designing your Personalizer architecture. In a production system, determining what factors affect the reward score can be challenging, and you may decide to change your reward score as your scenario or business needs change.

Code examples

These code snippets show you how to do the following tasks with the Personalizer client library for .NET:

Authenticate the client

In this section you'll do two things:

  • Specify your key and endpoint
  • Create a Personalizer client

Start by adding the following lines to your Program class. Make sure to add your key and endpoint from your Personalizer resource.

Important

Go to the Azure portal. If the Personalizer resource you created in the Prerequisites section deployed successfully, click the Go to Resource button under Next Steps. You can find your key and endpoint in the resource's key and endpoint page, under resource management.

Remember to remove the key from your code when you're done, and never post it publicly. For production, consider using a secure way of storing and accessing your credentials. For example, Azure key vault.

Important

Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like Azure Key Vault. See the Cognitive Services security article for more information.

private static readonly string ApiKey = "REPLACE-WITH-YOUR-PERSONALIZER-KEY";
private static readonly string ServiceEndpoint = "https://REPLACE-WITH-YOUR-PERSONALIZER-RESOURCE-NAME.cognitiveservices.azure.com";

Next, add a method to your program to create a new Personalizer client.

static PersonalizerClient InitializePersonalizerClient(string url)
{
    PersonalizerClient client = new PersonalizerClient(
        new ApiKeyServiceClientCredentials(ApiKey)) { Endpoint = url };

    return client;
}

Define actions and their features

Actions represent the choices from which Personalizer decides the best to use given the current context. The following code snippet creates a dictionary of four actions and the features (defined in the ActionFeatures class) that describe them. The function GetActions() assembles a list of RankableAction objects used in calling the Rank API for inference later in the quick-start.

Recall for our grocery website scenario, actions are the possible food items to be displayed in the "Featured Product" section on the website. The features here are simple examples that may be relevant in such a scenario.

Get food items as rankable actions

Actions represent the content choices from which you want Personalizer to select the best content item. Add the following methods to the Program class to represent the set of actions and their features.

static Dictionary<string, ActionFeatures> actions = new Dictionary<string, ActionFeatures>
{
    {"pasta", new ActionFeatures(
                    new BrandInfo(company: "pasta_inc"),
                    new ItemAttributes(
                        quantity: 1,
                        category: "Italian",
                        price: 12),
                    new DietaryAttributes(
                        vegan: false,
                        lowCarb: false,
                        highProtein: false,
                        vegetarian: false,
                        lowFat: true,
                        lowSodium: true))},
    {"bbq", new ActionFeatures(
                    new BrandInfo(company: "ambisco"),
                    new ItemAttributes(
                        quantity: 2,
                        category: "bbq",
                        price: 20),
                    new DietaryAttributes(
                        vegan: false,
                        lowCarb: true,
                        highProtein: true,
                        vegetarian: false,
                        lowFat: false,
                        lowSodium: false))},
    {"bao", new ActionFeatures(
                    new BrandInfo(company: "bao_and_co"),
                    new ItemAttributes(
                        quantity: 4,
                        category: "Chinese",
                        price: 8),
                    new DietaryAttributes(
                        vegan: false,
                        lowCarb: true,
                        highProtein: true,
                        vegetarian: false,
                        lowFat: true,
                        lowSodium: false))},
    {"hummus", new ActionFeatures(
                    new BrandInfo(company: "garbanzo_inc"),
                    new ItemAttributes(
                        quantity: 1,
                        category: "Breakfast",
                        price: 5),
                    new DietaryAttributes(
                        vegan: true,
                        lowCarb: false,
                        highProtein: true,
                        vegetarian: true,
                        lowFat: false,
                        lowSodium: false))},
    {"veg_platter", new ActionFeatures(
                    new BrandInfo(company: "farm_fresh"),
                    new ItemAttributes(
                        quantity: 1,
                        category: "produce",
                        price: 7),
                    new DietaryAttributes(
                        vegan: true,
                        lowCarb: true,
                        highProtein: false,
                        vegetarian: true,
                        lowFat: true,
                        lowSodium: true ))},
};

static IList<RankableAction> GetActions()
{
    IList<RankableAction> rankableActions = new List<RankableAction>();
    foreach (var action in actions)
    {
        rankableActions.Add(new RankableAction
        {
            Id = action.Key,
            Features = new List<object>() { action.Value }
        });
    }
    
    return rankableActions;
}

public class BrandInfo
{
    public string Company { get; set; }
    public BrandInfo(string company)
    {
        Company = company;
    }
}

public class ItemAttributes
{
    public int Quantity { get; set; }
    public string Category { get; set; }
    public double Price { get; set; }
    public ItemAttributes(int quantity, string category, double price)
    {
        Quantity = quantity;
        Category = category;
        Price = price;
    }
}

public class DietaryAttributes
{
    public bool Vegan { get; set; }
    public bool LowCarb { get; set; }
    public bool HighProtein { get; set; }
    public bool Vegetarian { get; set; }
    public bool LowFat { get; set; }
    public bool LowSodium { get; set; }
    public DietaryAttributes(bool vegan, bool lowCarb, bool highProtein, bool vegetarian, bool lowFat, bool lowSodium)
    {
        Vegan = vegan;
        LowCarb = lowCarb;
        HighProtein = highProtein;
        Vegetarian = vegetarian;
        LowFat = lowFat;
        LowSodium = lowSodium;

    }
}

public class ActionFeatures
{
    public BrandInfo BrandInfo { get; set; }
    public ItemAttributes ItemAttributes { get; set; }
    public DietaryAttributes DietaryAttributes { get; set; }
    public ActionFeatures(BrandInfo brandInfo, ItemAttributes itemAttributes, DietaryAttributes dietaryAttributes)
    {
        BrandInfo = brandInfo;
        ItemAttributes = itemAttributes;
        DietaryAttributes = dietaryAttributes;
    }
}

Define users and their context features

Context can represent the current state of your application, system, environment, or user. The following code creates a list with user preferences, and the GetContext() function selects a random user, random time of day, random location and random app type and assembles the features into a Context object, which will be used later when calling the Rank API. In our grocery website scenario, the user info consists of dietary preferences, a historical average of the order price. Let's assume our users are always on the move and include a location, time of day, and application type, which we choose randomly to simulate changing contexts every time GetContext() is called.

public static Context GetContext()
{
    return new Context(
            user: GetRandomUser(),
            timeOfDay: GetRandomTimeOfDay(),
            location: GetRandomLocation(),
            appType: GetRandomAppType());
}

static string[] timesOfDay = new string[] { "morning", "afternoon", "evening" };

static string[] locations = new string[] { "west", "east", "midwest" };

static string[] appTypes = new string[] { "edge", "safari", "edge_mobile", "mobile_app" };

static IList<UserProfile> users = new List<UserProfile>
{
    new UserProfile(
        name: "Bill",
        dietaryPreferences: new Dictionary<string, bool> { { "low_carb", true } },
        avgOrderPrice: "0-20"),
    new UserProfile(
        name: "Satya",
        dietaryPreferences: new Dictionary<string, bool> { { "low_sodium", true} },
        avgOrderPrice: "201+"),
    new UserProfile(
        name: "Amy",
        dietaryPreferences: new Dictionary<string, bool> { { "vegan", true }, { "vegetarian", true } },
        avgOrderPrice: "21-50")
};

static string GetRandomTimeOfDay()
{
    var random = new Random();
    var timeOfDayIndex = random.Next(timesOfDay.Length);
    Console.WriteLine($"TimeOfDay: {timesOfDay[timeOfDayIndex]}");
    return timesOfDay[timeOfDayIndex];
}

static string GetRandomLocation()
{
    var random = new Random();
    var locationIndex = random.Next(locations.Length);
    Console.WriteLine($"Location: {locations[locationIndex]}");
    return locations[locationIndex];
}

static string GetRandomAppType()
{
    var random = new Random();
    var appIndex = random.Next(appTypes.Length);
    Console.WriteLine($"AppType: {appTypes[appIndex]}");
    return appTypes[appIndex];
}

static UserProfile GetRandomUser()
{
    var random = new Random();
    var userIndex = random.Next(users.Count);
    Console.WriteLine($"\nUser: {users[userIndex].Name}");
    return users[userIndex];
}

public class UserProfile
{
    // Mark name as non serializable so that it is not part of the context features
    [NonSerialized()]
    public string Name;
    public Dictionary<string, bool> DietaryPreferences { get; set; }
    public string AvgOrderPrice { get; set; }

    public UserProfile(string name, Dictionary<string, bool> dietaryPreferences, string avgOrderPrice)
    {
        Name = name;
        DietaryPreferences = dietaryPreferences;
        AvgOrderPrice = avgOrderPrice;
    }
}

public class Context
{
    public UserProfile User { get; set; }
    public string TimeOfDay { get; set; }
    public string Location { get; set; }
    public string AppType { get; set; }

    public Context(UserProfile user, string timeOfDay, string location, string appType)
    {
        User = user;
        TimeOfDay = timeOfDay;
        Location = location;
        AppType = appType;
    }
}

The context features in this quick-start are simplistic, however, in a real production system, designing your features and evaluating their effectiveness can be non-trivial. You can refer to the aforementioned documentation for guidance.

Define a reward score based on user behavior

The reward score can be considered an indication how "good" the personalized action is. In a real production system, the reward score should be designed to align with your business objectives and KPIs. For example, your application code can be instrumented to detect a desired user behavior (for example, a purchase) that aligns with your business objective (for example, increased revenue).

In our grocery website scenario, we have three users: Bill, Satya, and Amy each with their own preferences. If a user purchases the product chosen by Personalizer, a reward score of 1.0 will be sent to the Reward API. Otherwise, the default reward of 0.0 will be used. In a real production system, determining how to design the reward may require some experimentation.

In the code below, the users' preferences and responses to the actions is hard-coded as a series of conditional statements, and explanatory text is included in the code for demonstrative purposes. In a real scenario, Personalizer will learn user preferences from the data sent in Rank and Reward API calls. You won't define these explicitly as in the example code.

The hard-coded preferences and reward can be succinctly described as:

  • Bill will purchase any product less than $10 as long as he isn't in the midwest.
  • When Bill is in the midwest, he'll purchase any product as long as it's low in carbs.
  • Satya will purchase any product that's low in sodium.
  • Amy will purchase any product that's either vegan or vegetarian.
public static float GetRewardScore(Context context, string actionId)
{
    float rewardScore = 0.0f;
    string userName = context.User.Name;
    ActionFeatures actionFeatures = actions[actionId];
    if (userName.Equals("Bill"))
    {
        if (actionFeatures.ItemAttributes.Price < 10 && !context.Location.Equals("midwest"))
        {
            rewardScore = 1.0f;
            Console.WriteLine($"\nBill likes to be economical when he's not in the midwest visiting his friend Warren. He bought {actionId} because it was below a price of $10.");
        }
        else if (actionFeatures.DietaryAttributes.LowCarb && context.Location.Equals("midwest"))
        {
            rewardScore = 1.0f;
            Console.WriteLine($"\nBill is visiting his friend Warren in the midwest. There he's willing to spend more on food as long as it's low carb, so Bill bought {actionId}.");
        }
        else if (actionFeatures.ItemAttributes.Price >= 10 && !context.Location.Equals("midwest"))
        {
            rewardScore = 1.0f;
            Console.WriteLine($"\nBill didn't buy {actionId} because the price was too high when not visting his friend Warren in the midwest.");
        }
        else if (actionFeatures.DietaryAttributes.LowCarb && context.Location.Equals("midwest"))
        {
            rewardScore = 1.0f;
            Console.WriteLine($"\nBill didn't buy {actionId} because it's not low-carb, and he's in the midwest visitng his friend Warren.");
        }
    }
    else if (userName.Equals("Satya"))
    {
        if (actionFeatures.DietaryAttributes.LowSodium)
        {
            rewardScore = 1.0f;
            Console.WriteLine($"\nSatya is health conscious, so he bought {actionId} since it's low in sodium.");
        }
        else
        {
            Console.WriteLine($"\nSatya did not buy {actionId} because it's not low sodium.");
        }
    }
    else if (userName.Equals("Amy"))
    {
        if (actionFeatures.DietaryAttributes.Vegan || actionFeatures.DietaryAttributes.Vegetarian)
        {
            rewardScore = 1.0f;
            Console.WriteLine($"\nAmy likes to eat plant-based foods, so she bought {actionId} because it's vegan or vegetarian friendly.");
        }
        else
        {
            Console.WriteLine($"\nAmy did not buy {actionId} because it's not vegan or vegetarian.");
        }
    }
    return rewardScore;
}

Run Rank and Reward calls for each user

A Personalizer event cycle consists of Rank and Reward API calls. In our grocery website scenario, each Rank call is made to determine which product should be displayed in the "Featured Product" section. Then the Reward call tells Personalizer whether or not the featured product was purchased by the user.

Request the best action

In a Rank call, you need to provide at least two arguments: a list of RankableActions (actions and their features), and a list of (context) features. The response will include the RewardActionId, which is the ID of the action Personalizer has determined is best for the given context. The response also includes the EventId, which is needed in the Reward API so Personalize knows how to link the data from the corresponding Rank and Reward calls. For more information, refer to the Rank API docs.

Send a reward

In a Reward call, you need to provide two arguments: the EventId, which links the Rank and Reward calls to the same unique event, and the value (reward score). Recall that the reward score is a signal that tells Personalizer if the decision made in the Rank call was a good or not. A reward score is typically a number between 0.0 and 1.0. It's worth reiterating that determining how to design the reward can be non-trivial.

Run a Rank and Reward cycle

The following code loops through a single cycle of Rank and Reward calls for a randomly selected user, then prints relevant information to the console at each step.

static void Main(string[] args)
{
    int iteration = 1;
    bool runLoop = true;

    // Get the actions list to choose from personalizer with their features.
    IList<RankableAction> actions = GetActions();

    // Initialize Personalizer client.
    PersonalizerClient client = InitializePersonalizerClient(ServiceEndpoint);

    do
    {
        Console.WriteLine("\nIteration: " + iteration++);

        // <rank>
        // Get context information.
        Context context = GetContext();

        // Create current context from user specified data.
        IList<object> currentContext = new List<object>() {
            context
        };

        // Generate an ID to associate with the request.
        string eventId = Guid.NewGuid().ToString();

        // Rank the actions
        var request = new RankRequest(actions: actions, contextFeatures: currentContext, eventId: eventId);
        RankResponse response = client.Rank(request);
        // </rank>

        Console.WriteLine($"\nPersonalizer service thinks {context.User.Name} would like to have: {response.RewardActionId}.");

        // <reward>
        float reward = GetRewardScore(context, response.RewardActionId);

        // Send the reward for the action based on user response.
        client.Reward(response.EventId, new RewardRequest(reward));
        // </reward>

        Console.WriteLine("\nPress q to break, any other key to continue:");
        runLoop = !(GetKey() == "Q");

    } while (runLoop);
}

private static string GetKey()
{
    return Console.ReadKey().Key.ToString().Last().ToString().ToUpper();
}

Run the program

Run the application with the dotnet run command from your application directory.

dotnet run

Generate sample events for analysis (Optional)

You can easily generate 5,000 events from this quickstart demo scenario, which is sufficient to get experience with using Apprentice mode, Online mode, running offline evaluations, and creating feature evaluations. Simply replace the Main() method of the above code in the Run a Rank and Reward cycle section with:


   static void Main(string[] args)
{
    int iteration = 1;
    int runLoop = 0;

    // Get the actions list to choose from personalizer with their features.
    IList<RankableAction> actions = GetActions();

    // Initialize Personalizer client.
    PersonalizerClient client = InitializePersonalizerClient(ServiceEndpoint);

    do
    {
        Console.WriteLine("\nIteration: " + iteration++);

        // <rank>
        // Get context information.
        Context context = GetContext();

        // Create current context from user specified data.
        IList<object> currentContext = new List<object>() {
            context
        };

        // Generate an ID to associate with the request.
        string eventId = Guid.NewGuid().ToString();

        // Rank the actions
        var request = new RankRequest(actions: actions, contextFeatures: currentContext, eventId: eventId);
        RankResponse response = client.Rank(request);
        // </rank>

        Console.WriteLine($"\nPersonalizer service thinks {context.User.Name} would like to have: {response.RewardActionId}.");

        // <reward>
        float reward = GetRewardScore(context, response.RewardActionId);

        // Send the reward for the action based on user response.
        client.Reward(response.EventId, new RewardRequest(reward));
        // </reward>

        runLoop = runLoop + 1;

    } while (runLoop < 1000);
}

then run the program.

The quickstart program asks a couple of questions to gather user preferences, known as features, then provides the top action.

The source code for this quickstart is available.

You'll need to install the Personalizer client library forN ode.js to:

  • Authenticate the quickstart example client with a Personalizer resource in Azure.
  • Send context and action features to the Reward API, which will return the best action from the Personalizer model
  • Send a reward score to the Rank API and train the Personalizer model.

Reference documentation |Library source code | Package (npm) | Quickstart code sample

Prerequisites

  • Azure subscription - Create one for free
  • Install Node.js and npm (verified with Node.js v14.16.0 and npm 6.14.11).
  • Once you have your Azure subscription, create a Personalizer resource in the Azure portal to get your key and endpoint. After it deploys, select Go to resource.
    • You'll need the key and endpoint from the resource you create to connect your application to the Personalizer API. You'll paste your key and endpoint into the code below later in the quickstart.
    • You can use the free pricing tier (F0) to try the service, and upgrade later to a paid tier for production.

Setting Up

Change the model update frequency

In the Azure portal, in the Personalizer resource on the Configuration page, change the Model update frequency to 30 seconds. This short duration will train the service rapidly, allowing you to see how the top action changes for each iteration.

Change model update frequency

When a Personalizer loop is first instantiated, there is no model since there has been no Reward API calls to train from. Rank calls will return equal probabilities for each item. Your application should still always rank content using the output of RewardActionId.

Change the reward wait time

In the Azure portal, in the Personalizer resource on the Configuration page, change the Reward wait time to 10 minutes. This short duration will train the service rapidly, allowing you to see how the top action changes for each iteration.

Change reward wait time

Create a new Node.js application

In a console window (such as cmd, PowerShell, or Bash), create a new directory for your app, and navigate to it.

mkdir myapp && cd myapp

Run the npm init -y command to create a package.json file.

npm init -y

Create a new Node.js application in your preferred editor or IDE named sample.js and create variables for your resource's endpoint and subscription key.

Important

Go to the Azure portal. If the Personalizer resource you created in the Prerequisites section deployed successfully, click the Go to Resource button under Next Steps. You can find your key and endpoint in the resource's key and endpoint page, under resource management.

Remember to remove the key from your code when you're done, and never post it publicly. For production, consider using a secure way of storing and accessing your credentials. For example, Azure key vault.

Important

Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure way of storing and accessing your credentials like Azure Key Vault. For more information about security see the Cognitive Services security article.

const uuidv1 = require('uuid/v1');
const Personalizer = require('@azure/cognitiveservices-personalizer');
const CognitiveServicesCredentials = require('@azure/ms-rest-azure-js').CognitiveServicesCredentials;
const readline = require('readline-sync');

// The key specific to your personalization service instance; e.g. "0123456789abcdef0123456789ABCDEF"
const serviceKey = "<REPLACE-WITH-YOUR-PERSONALIZER-KEY>";

// The endpoint specific to your personalization service instance; 
// e.g. https://<your-resource-name>.cognitiveservices.azure.com
const baseUri = "https://<REPLACE-WITH-YOUR-PERSONALIZER-ENDPOINT>.cognitiveservices.azure.com";

Install the Node.js library for Personalizer

Install the Personalizer client library for Node.js with the following command:

npm install @azure/cognitiveservices-personalizer --save

Install the remaining npm packages for this quickstart:

npm install @azure/ms-rest-azure-js @azure/ms-rest-js readline-sync uuid --save

Object model

The Personalizer client is a PersonalizerClient object that authenticates to Azure using Microsoft.Rest.ServiceClientCredentials, which contains your key.

To ask for the single best item of the content, create a RankRequest, then pass it to client.Rank method. The Rank method returns a RankResponse.

To send a reward to Personalizer, create a RewardRequest, then pass it to the Reward method on the Events class.

The reward scoring in this quickstart is trivial. In a production system, the determination of what impacts the reward score and by how much can be a complex process, that you may decide to change over time. Reward scores should be one of the primary design decisions in your Personalizer architecture.

Code examples

These code snippets show you how to do the following tasks with the Personalizer client library for Node.js:

Authenticate the client

Instantiate the PersonalizerClient with your serviceKey and baseUri that you created earlier.

const credentials = new CognitiveServicesCredentials(serviceKey);

// Initialize Personalization client.
const personalizerClient = new Personalizer.PersonalizerClient(credentials, baseUri);

Get content choices represented as actions

Actions represent the content choices from which you want Personalizer to select the best content item. Add the following methods to the Program class to represent the set of actions and their features.

function getContextFeaturesList() {
  const timeOfDayFeatures = ['morning', 'afternoon', 'evening', 'night'];
  const tasteFeatures = ['salty', 'sweet'];

  let answer = readline.question("\nWhat time of day is it (enter number)? 1. morning 2. afternoon 3. evening 4. night\n");
  let selection = parseInt(answer);
  const timeOfDay = selection >= 1 && selection <= 4 ? timeOfDayFeatures[selection - 1] : timeOfDayFeatures[0];

  answer = readline.question("\nWhat type of food would you prefer (enter number)? 1. salty 2. sweet\n");
  selection = parseInt(answer);
  const taste = selection >= 1 && selection <= 2 ? tasteFeatures[selection - 1] : tasteFeatures[0];

  console.log("Selected features:\n");
  console.log("Time of day: " + timeOfDay + "\n");
  console.log("Taste: " + taste + "\n");

  return [
    {
      "time": timeOfDay
    },
    {
      "taste": taste
    }
  ];
}
function getActionsList() {
  return [
    {
      "id": "pasta",
      "features": [
        {
          "taste": "salty",
          "spiceLevel": "medium"
        },
        {
          "nutritionLevel": 5,
          "cuisine": "italian"
        }
      ]
    },
    {
      "id": "ice cream",
      "features": [
        {
          "taste": "sweet",
          "spiceLevel": "none"
        },
        {
          "nutritionalLevel": 2
        }
      ]
    },
    {
      "id": "juice",
      "features": [
        {
          "taste": "sweet",
          "spiceLevel": "none"
        },
        {
          "nutritionLevel": 5
        },
        {
          "drink": true
        }
      ]
    },
    {
      "id": "salad",
      "features": [
        {
          "taste": "salty",
          "spiceLevel": "low"
        },
        {
          "nutritionLevel": 8
        }
      ]
    }
  ];
}

Create the learning loop

The Personalizer learning loop is a cycle of Rank and Reward calls. In this quickstart, each Rank call, to personalize the content, is followed by a Reward call to tell Personalizer how well the service performed.

The following code loops through a cycle of asking the user their preferences at the command line, sending that information to Personalizer to select the best action, presenting the selection to the customer to choose from among the list, and then sending a reward to Personalizer signaling how well the service did in its selection.

let runLoop = true;

do {

  let rankRequest = {}

  // Generate an ID to associate with the request.
  rankRequest.eventId = uuidv1();

  // Get context information from the user.
  rankRequest.contextFeatures = getContextFeaturesList();

  // Get the actions list to choose from personalization with their features.
  rankRequest.actions = getActionsList();

  // Exclude an action for personalization ranking. This action will be held at its current position.
  rankRequest.excludedActions = getExcludedActionsList();

  rankRequest.deferActivation = false;

  // Rank the actions
  const rankResponse = await personalizerClient.rank(rankRequest);

  console.log("\nPersonalization service thinks you would like to have:\n")
  console.log(rankResponse.rewardActionId);

  // Display top choice to user, user agrees or disagrees with top choice
  const reward = getReward();

  console.log("\nPersonalization service ranked the actions with the probabilities as below:\n");
  for (let i = 0; i < rankResponse.ranking.length; i++) {
    console.log(JSON.stringify(rankResponse.ranking[i]) + "\n");
  }

  // Send the reward for the action based on user response.

  const rewardRequest = {
    value: reward
  }

  await personalizerClient.events.reward(rankRequest.eventId, rewardRequest);

  runLoop = continueLoop();

} while (runLoop);

Take a closer look at the rank and reward calls in the following sections.

Add the following methods, which get the content choices, before running the code file:

  • getActionsList
  • getContextFeaturesList

Request the best action

To make a Rank request, you'll need to provide: a list of 'RankActions' (actions), a list of context features (context), an optional list of actions (excludeActions) to remove from consideration by Personalizer, and a unique event ID to receive the response.

This quickstart has simple context features of time of day and user food preference. In production systems, determining and evaluating actions and features can be a non-trivial matter.

let rankRequest = {}

// Generate an ID to associate with the request.
rankRequest.eventId = uuidv1();

// Get context information from the user.
rankRequest.contextFeatures = getContextFeaturesList();

// Get the actions list to choose from personalization with their features.
rankRequest.actions = getActionsList();

// Exclude an action for personalization ranking. This action will be held at its current position.
rankRequest.excludedActions = getExcludedActionsList();

rankRequest.deferActivation = false;

// Rank the actions
const rankResponse = await personalizerClient.rank(rankRequest);

Send a reward

To get the reward score to send in the Reward request, the program gets the user's selection from the command line, assigns a numeric value to the selection, then sends the unique event ID and the reward score as the numeric value to the Reward API.

This quickstart assigns a simple number as a reward score, either a zero or a 1. In production systems, determining when and what to send to the Reward call can be a non-trivial matter, depending on your specific needs.

const rewardRequest = {
  value: reward
}

await personalizerClient.events.reward(rankRequest.eventId, rewardRequest);

Run the program

Run the application with the Node.js from your application directory.

node sample.js

The quickstart program asks a couple of questions to gather user preferences, known as features, then provides the top action.

To start, you'll need to install the Personalizer client library for Python to:

  • Authenticate the quickstart example client with your Personalizer resource in Azure.
  • Send context and action features to the Rank API, which will return the best action from Personalizer.
  • Send a reward score to the Reward API to train the Personalizer model.

Reference documentation | Library source code | Package (pypi) | Quickstart code sample

Prerequisites

  • Azure subscription - Create one for free
  • Python 3.x
  • Once your Azure subscription is set up, create a Personalizer resource in the Azure portal and obtain your key and endpoint. After it deploys, click Go to resource.
    • You'll need the key and endpoint from the created resource to connect your application to the Personalizer API, which you'll paste into the quick-start code below.
    • You can use the free pricing tier (F0) to try the service, then upgrade to a paid tier for production at a later time.

Setting Up

Change the model update frequency

In the Azure portal, in the Personalizer resource on the Configuration page, change the Model update frequency to 30 seconds. This short duration will train the service rapidly, allowing you to see how the top action changes for each iteration.

Change model update frequency

When a Personalizer loop is first instantiated, there is no model since there has been no Reward API calls to train from. Rank calls will return equal probabilities for each item. Your application should still always rank content using the output of RewardActionId.

Change the reward wait time

In the Azure portal, in the Personalizer resource on the Configuration page, change the Reward wait time to 10 minutes. This short duration will train the service rapidly, allowing you to see how the top action changes for each iteration.

Change reward wait time

Install the client library

After installing Python, install the Personalizer client library with pip:

pip install azure-cognitiveservices-personalizer

Create a new Python application

Create a new Python application file named "personalizer_quickstart.py". This application will handle both the example scenario logic and calls the Personalizer APIs.

In the file, create variables for your resource's endpoint and subscription key.

Important

Go to the Azure portal. If the Personalizer resource you created in the Prerequisites section deployed successfully, click the Go to Resource button under Next Steps. You can find your key and endpoint in the resource's key and endpoint page, under resource management.

Remember to remove the key from your code when you're done, and never post it publicly. For production, consider using a secure way of storing and accessing your credentials. For example, Azure key vault.

Important

Remember to remove the key from your code when you're done, and never post it publicly. For production, use a secure method to store and access your credentials like Azure Key Vault. For more information, see the Cognitive Services security.

from azure.cognitiveservices.personalizer import PersonalizerClient
from azure.cognitiveservices.personalizer.models import RankableAction, RewardRequest, RankRequest
from msrest.authentication import CognitiveServicesCredentials

import datetime, json, os, time, uuid, random

key = "<paste-your-personalizer-key-here>"
endpoint = "<paste-your-personalizer-endpoint-here>"

Object model

The client is a PersonalizerClient object that authenticates to Azure using Microsoft.Rest.ServiceClientCredentials containing your key.

To request the best action from Personalizer, create a RankRequest containing the set of RankableActions that Personalizer will choose from, and a set of context features. The RankRequest will be passed to the client.Rank method, which returns a RankResponse containing an event ID.

To send a reward score to Personalizer, use the Reward method in the EventOperations class and include the event ID corresponding to the Rank call that returned the best action, and the reward score.

Later in this quick-start, we'll define an example reward score. However, the reward is one of the most important considerations when designing your Personalizer architecture. In a production system, determining what factors affect the reward score can be challenging, and you may decide to change your reward score as your scenario or business needs change.

Code examples

These code snippets demonstrate how to use the Personalizer client library for Python to:

Authenticate the client

Instantiate the PersonalizerClient with the key and endpoint you obtained from your Personalizer resource.

# Instantiate a Personalizer client
client = PersonalizerClient(endpoint, CognitiveServicesCredentials(key))

Define actions and their features

Actions represent the choices from which Personalizer decides the best to use given the current context. The following code snippet creates a dictionary of four actions and the features that describe them. The function get_actions() assembles a list of RankableAction objects used in calling the Rank API for inference later in the quick-start.

Recall for our grocery website scenario, actions are the possible food items to be displayed in the "Featured Product" section on the website. The features here are simple examples that may be relevant in such a scenario.

actions_and_features = {
    'pasta': {
        'brand_info': {
            'company':'pasta_inc'
        }, 
        'attributes': {
            'qty':1, 'cuisine':'italian',
            'price':12
        },
        'dietary_attributes': {
            'vegan': False,
            'low_carb': False,
            'high_protein': False,
            'vegetarian': False,
            'low_fat': True,
            'low_sodium': True
        }
    },
    'bbq': {
        'brand_info' : {
            'company': 'ambisco'
        },
        'attributes': {
            'qty': 2,
            'category': 'bbq',
            'price': 20
        }, 
        'dietary_attributes': {
            'vegan': False,
            'low_carb': True,
            'high_protein': True,
            'vegetarian': False,
            'low_fat': False,
            'low_sodium': False
        }
    },
    'bao': {
        'brand_info': {
            'company': 'bao_and_co'
        },
        'attributes': {
            'qty': 4,
            'category': 'chinese',
            'price': 8
        }, 
        'dietary_attributes': {
            'vegan': False,
            'low_carb': True,
            'high_protein': True,
            'vegetarian': False,
            'low_fat': True,
            'low_sodium': False
        }
    },
    'hummus': {
        'brand_info' : { 
            'company': 'garbanzo_inc'
        },
        'attributes' : {
            'qty': 1,
            'category': 'breakfast',
            'price': 5
        }, 
        'dietary_attributes': {
            'vegan': True, 
            'low_carb': False,
            'high_protein': True,
            'vegetarian': True,
            'low_fat': False, 
            'low_sodium': False
        }
    },
    'veg_platter': {
        'brand_info': {
            'company': 'farm_fresh'
        }, 
        'attributes': {
            'qty': 1,
            'category': 'produce', 
            'price': 7
        },
        'dietary_attributes': {
            'vegan': True,
            'low_carb': True,
            'high_protein': False,
            'vegetarian': True,
            'low_fat': True,
            'low_sodium': True
        }
    }
}

def get_actions():
    res = []
    for action_id, feat in actions_and_features.items():
        action = RankableAction(id=action_id, features=[feat])
        res.append(action)
    return res

Define users and their context features

Context can represent the current state of your application, system, environment, or user. The following code creates a dictionary with user preferences, and the get_context() function assembles the features into a list for one particular user, which will be used later when calling the Rank API. In our grocery website scenario, the context consists of dietary preferences, a historical average of the order price. Let's assume our users are always on the move and include a location, time of day, and application type, which we choose randomly to simulate changing contexts every time get_context() is called. Finally, get_random_users() creates a random set of 5 user names from the user profiles, which will be used to simulate Rank/Reward calls later on.

user_profiles = {
    'Bill': {
        'dietary_preferences': 'low_carb', 
        'avg_order_price': '0-20',
        'browser_type': 'edge'
    },
    'Satya': {
        'dietary_preferences': 'low_sodium',
        'avg_order_price': '201+',
        'browser_type': 'safari'
    },
    'Amy': {
        'dietary_preferences': {
            'vegan', 'vegetarian'
        },
        'avg_order_price': '21-50',
        'browser_type': 'edge'},
    }

def get_context(user):
    location_context = {'location': random.choice(['west', 'east', 'midwest'])}
    time_of_day = {'time_of_day': random.choice(['morning', 'afternoon', 'evening'])}
    app_type = {'application_type': random.choice(['edge', 'safari', 'edge_mobile', 'mobile_app'])}
    res = [user_profiles[user], location_context, time_of_day, app_type]
    return res

def get_random_users(k = 5):
    return random.choices(list(user_profiles.keys()), k=k)

The context features in this quick-start are simplistic, however, in a real production system, designing your features and evaluating their effectiveness can be non-trivial. You can refer to the aforementioned documentation for guidance.

Define a reward score based on user behavior

The reward score can be considered an indication how "good" the personalized action is. In a real production system, the reward score should be designed to align with your business objectives and KPIs. For example, your application code can be instrumented to detect a desired user behavior (for example, a purchase) that aligns with your business objective (for example, increased revenue).

In our grocery website scenario, we have three users: Bill, Satya, and Amy each with their own preferences. If a user purchases the product chosen by Personalizer, a reward score of 1.0 will be sent to the Reward API. Otherwise, the default reward of 0.0 will be used. In a real production system, determining how to design the reward may require some experimentation.

In the code below, the users' preferences and responses to the actions is hard-coded as a series of conditional statements, and explanatory text is included in the code for demonstrative purposes. In a real scenario, Personalizer will learn user preferences from the data sent in Rank and Reward API calls. You won't define these explicitly as in the example code.

The hard-coded preferences and reward can be succinctly described as:

  • Bill will purchase any product less than $10 as long as he isn't in the midwest.
  • When Bill is in the midwest, he'll purchase any product as long as it's low in carbs.
  • Satya will purchase any product that's low in sodium.
  • Amy will purchase any product that's either vegan or vegetarian.
def get_reward_score(user, actionid, context):
    reward_score = 0.0
    action = actions_and_features[actionid]
    
    if user == 'Bill':
        if action['attributes']['price'] < 10 and (context[1]['location'] !=  "midwest"):
            reward_score = 1.0
            print("Bill likes to be economical when he's not in the midwest visiting his friend Warren. He bought", actionid, "because it was below a price of $10.")
        elif (action['dietary_attributes']['low_carb'] == True) and (context[1]['location'] ==  "midwest"):
            reward_score = 1.0
            print("Bill is visiting his friend Warren in the midwest. There he's willing to spend more on food as long as it's low carb, so Bill bought" + actionid + ".")
            
        elif (action['attributes']['price'] >= 10) and (context[1]['location'] != "midwest"):
            print("Bill didn't buy", actionid, "because the price was too high when not visting his friend Warren in the midwest.")
            
        elif (action['dietary_attributes']['low_carb'] == False) and (context[1]['location'] ==  "midwest"):
            print("Bill didn't buy", actionid, "because it's not low-carb, and he's in the midwest visitng his friend Warren.")
             
    elif user == 'Satya':
        if action['dietary_attributes']['low_sodium'] == True:
            reward_score = 1.0
            print("Satya is health conscious, so he bought", actionid,"since it's low in sodium.")
        else:
            print("Satya did not buy", actionid, "because it's not low sodium.")   
            
    elif user == 'Amy':
        if (action['dietary_attributes']['vegan'] == True) or (action['dietary_attributes']['vegetarian'] == True):
            reward_score = 1.0
            print("Amy likes to eat plant-based foods, so she bought", actionid, "because it's vegan or vegetarian friendly.")       
        else:
            print("Amy did not buy", actionid, "because it's not vegan or vegetarian.")
                
    return reward_score

Run Rank and Reward calls for each user

A Personalizer event cycle consists of Rank and Reward API calls. In our grocery website scenario, each Rank call is made to determine which product should be displayed in the "Featured Product" section. Then the Reward call tells Personalizer whether or not the featured product was purchased by the user.

Request the best action

In a Rank call, you need to provide at least two arguments: a list of RankableActions (actions and their features), and a list of (context) features. The response will include the reward_action_id, which is the ID of the action Personalizer has determined is best for the given context. The response also includes the event_id, which is needed in the Reward API so Personalize knows how to link the data from the Reward and Rank calls. For more information, refer to the Rank API docs.

Send a reward

In a Reward call, you need to provide two arguments: the event_id, which links the Reward and Rank calls to the same unique event, and the value (reward score). Recall that the reward score is a signal that tells Personalizer if the decision made in the Rank call was a good or not. A reward score is typically a number between 0.0 and 1.0. It's worth reiterating that determining how to design the reward can be non-trivial.

Run a Rank and Reward cycle

The following code loops through five cycles of Rank and Reward calls for a randomly selected set of example users, then prints relevant information to the console at each step.

def run_personalizer_cycle():
    actions = get_actions()
    user_list = get_random_users()
    for user in user_list:
        print("------------")
        print("User:", user, "\n")
        context = get_context(user)
        print("Context:", context, "\n")
        
        rank_request = RankRequest(actions=actions, context_features=context)
        response = client.rank(rank_request=rank_request)
        print("Rank API response:", response, "\n")
        
        eventid = response.event_id
        actionid = response.reward_action_id
        print("Personalizer recommended action", actionid, "and it was shown as the featured product.\n")
        
        reward_score = get_reward_score(user, actionid, context)
        client.events.reward(event_id=eventid, value=reward_score)     
        print("\nA reward score of", reward_score , "was sent to Personalizer.")
        print("------------\n")

continue_loop = True
while continue_loop:
    run_personalizer_cycle()
    
    br = input("Press Q to exit, or any other key to run another loop: ")
    if(br.lower()=='q'):
        continue_loop = False

Run the program

Once all the above code is included in your Python file, you can run it from your application directory.

python personalizer_quickstart.py

Generate sample events for analysis (Optional)

You can easily generate 5,000 events from this quickstart demo scenario, which is sufficient to get experience with using Apprentice mode, Online mode, running offline evaluations, and creating feature evaluations. Simply replace this code from above:

continue_loop = True
while continue_loop:
    run_personalizer_cycle()
    
    br = input("Press Q to exit, or any other key to run another loop: ")
    if(br.lower()=='q'):
        continue_loop = False

with the following:

for i in range(0,1000):
    run_personalizer_cycle()

then run the program.

The quickstart program asks a couple of questions to gather user preferences, known as features, then provides the top action.

Clean up resources

To clean up your Cognitive Services subscription, you can delete the resource or the resource group, which will delete any associated resources.

Download the quickstart trained model

If you'd like download a Personalizer model that has been trained on 5,000 events from the QuickStart example, you can visit the Azure-Samples repository and download the model zip file, then upload this to your Personalizer instance under the "Setup" -> "Model Import/Export" section.

Next steps