---

March 2017

Volume 32 Number 3

[Bot Framework]

Making Bots More Intelligent

By Kevin Ashley | March 2017

Microsoft Bot Framework, LUIS, Azure Bot Service, Azure Functions I hear it everywhere these days: bots are the new apps. One reason is the ease and efficiency they bring to common tasks. Think of it: Instead of jumping from Bing or Google to two or three Web sites or more to book a flight or make reservations at a restaurant, you’re able to simply ask some kind of digital assistant to do it for you. But there may be an even more significant reason for app developers to embrace this trend: Mobile apps are becoming more and more expensive to create. Building just one mobile app takes anywhere between 30,000 and half a million dollars for each platform: iOS, Android and Windows.

True, there are some cross-platform technologies—such as Unity or Xamarin—that can help mitigate the cost of developing for multiple platforms. But bots are emerging as another option, shifting the paradigm by employing a conversational interface. This approach leverages communications used by humanity for thousands of years and spans voice and text (think Skype or Messenger). The Conversational UI (CUI) can also be enhanced with images, videos and traditional controls such as Buttons and CardActions. In bots, the focus is on simplicity and the natural aspects of input, rather than crafting the UI with traditional controls.

The benefit? A potential aggregate savings in the billions of dollars as organizations move away from building distinct UIs for each screen paradigm and instead adopt conversational interfaces that work across all devices and form factors, with a screen or without. For example, I built an Active Fitness bot that helps you find running, cycling, hiking, skiing and other trails near you. (Visit bit.ly/2knZVbr to add to Skype.) While the app features a rich UI and controls, the bot is designed to understand your conversational input, as shown in Figure 1.

Figure 1 Active Fitness Bot That Shows You Trails

The impact extends beyond app development. Bots can address the challenges of Web sites that are becoming increasingly difficult to navigate, with many using old or outdated Web frameworks or standards. Try to complete a task at any customer service Web site, for example, and you’ll probably spend a good half hour trying to navigate the Web site menu.

And then there’s the unique reach of bots. Imagine a card game app, such as Solitaire. Before bots, I had to build a version for each platform. Now, however, bots disrupt the trap that is platform lockdown. Suddenly, my Windows-based apps work as bots on all my friends’ iPhones and Androids. Why? Because bots work in channels—Skype, Telegram, Facebook Messenger, Slack and other ubiquitous app platforms.

I figured it’d be cool to build the most popular game in the world using Microsoft Bot Framework, so Figure 2 shows my take on Solitaire, all ready to play. (Visit bit.ly/2jrzP7S to add to Skype.)

Figure 2 My Solitaire Bot on Skype Handles 500,000 Requests per Week

To start creating bots, go to the Microsoft Bot Framework site (dev.botframework.com). You can build bots using the Microsoft .NET Framework or Node.js, whichever technology you prefer. Recently, Microsoft added Azure Bot Service (bit.ly/2knEtU6) as yet another convenient way to implement bots. Azure Bot Service provides a browser-based experience for building bots from easy-to-use templates. The service uses Azure Functions, an event-based serverless code architecture with “functions” that allow you to save money and scale your bots better. 

Here’s what’s great about Azure Functions: When I started building bots, I realized that each time I needed a bot I’d have to create a new Web app to provide an app container. That’s fine for one app, but not for multiple bots, which would each require a separate infrastructure. Instead, Azure Functions allows you to host small pieces of code in the cloud, without worrying about the whole application infrastructure.

Intelligence and Bot Development

Just like apps, bots are here to help you with a specific task they can do best. For example, my Solitaire bot can play Solitaire with you and maybe even teach you rules and tricks of the game. True, it’s not likely to pass the Turing test (a way to measure a machine’s intelligence developed by Alan Turing in the 1950s), but that would probably be overkill for most bots.

Likewise, a bot for your local motor vehicle office should know certain things about submitting a payment for your license registration or how to help you pay a ticket or submit a driver license extension. A weather bot needs to answer weather-related questions, and the intelligence of the bot might include ways to figure out location and user needs for specific weather data.

Bots can automate many things via a conversational interface, but they can also be enhanced with images, buttons and other controls that make interaction even more convenient with mobile devices. Bot intelligence really comes into play when conversations become open-ended. Using dialogs for a predefined set of answers is a standard, relatively easy programming task. Responding to a human conversation, on the other hand, implies intelligence. Fortunately, Microsoft Bot Framework allows you to do both: LuisDialogs enables support for natural language-based conversations, while more traditional controls let users press buttons, carousels or other controls to select options.

Intelligence is at the heart of the conversational UI employed by many bots, enabling intuitive interaction with human users. The growing number of Cognitive Services from Microsoft (bit.ly/2jx1kMQ) allows you to plug intelligent capabilities right into your bot. These services include (but are not limited to):

• Vision
• Speech
• Language
• Knowledge
• Search
• Location

Check out Alessandro Del Sole’s fantastic article on Cognitive Services at msdn.com/magazine/mt742868. 

Making your bot more intelligent involves the use of some common measures that might not make it through a Turing test, but will certainly improve conversations with the bot:

• Dialogs, images and other ways to gather normalized data, convey a message and simplify conversations
• Localization
• Help and dialog pipelines to improve basic bot functions
• User data (such as location) to eliminate duplicating questions

You don’t want to be too ambitious and try to create a bot that resembles Douglas Adams’ city-sized Deep Thought computer (from “The Hitchhiker’s Guide to the Galaxy”). Rather, when I started building my bots, I limited them to functions that were essential to primary tasks—like playing a card game, or suggesting best trails to run—and then enhanced them with intelligence.

Language Intelligence with LUIS

The first intelligence component to consider adding to a conversational interface is Microsoft Language Understanding Intelligence Service (LUIS). This service works by analyzing conversations or sentences you send to it, and extracting actual intents and entities specific to your application. To add a LUIS dialog to your bot, simply create a class that inherits Luis­Dialog, register your LUIS app at luis.aiand provide app id and key to your dialog:

[LuisModel("<YOUR_LUIS_APP_ID>", "<YOUR_LUIS_APP_KEY>")]
[Serializable]
public class IntelligentLanguageDialog : LuisDialog<object>

For more in-depth LUIS guidance, please check out Ashish Sahu’s January 20176 MSDN Magazine article at msdn.com/magazine/mt745095.

In my Active Fitness Trails bot (Figure 3) I defined intents related to fitness activities, and I expect the user to provide location and fitness activity as part of an intelligent bot conversation. I want a user to be able to say, for example, “Show me ski trails in Colorado,” and the bot should understand this and come up with a set of trails. Users of my Active Fitness app run millions of trails daily, and by querying Active Fitness they can map the GetActivityLocation intent from my LUIS model, which returns geography and activity entities back to the bot.

Figure 3 Model in LUIS

Imagine how much coding I’d have to do to extract all that data from non-normalized user input! Even a short human sentence often involves a level of complexity that only machine learning tools can understand and process. Add all possible variations of that simple sentence and you realize that bot intelligence is a non-trivial task. Fortunately, LUIS acts as an intelligent conversation analyzer and extractor that does all the hard work for you. All you need is to add it to your bot.

Once the model is defined, I can train it by providing utterances that help LUIS understand what my intents are and how to get entities for my bot. Training the model for my GetActivity­Location intent, I provided the following utterances: “Show cycling trails nearby,” “Show latest snowboard trails in Austria,” and, “Where are the longest ski trails in Utah?” As Figure 4 shows, when LUIS processes entities, it highlights entities I specified in my model. What if LUIS doesn’t recognize an entity? This is where you train it by manually mapping entities. For example, I had to train my bot to recognize “snowboard” as an activity.

Figure 4 Model Training in LUIS

Next, you define methods that handle intents returned by your LUIS model. You can have methods that handle a None intent or any other intents from your model, as Figure 5 shows.

Figure 5 Handling Intents

[LuisIntent("None")]
  public async Task NoneIntent(IDialogContext context, LuisResult result)
  {
    await context.PostAsync(
      $"You have reached the none intent. You said: {result.Query}"); //
    context.Wait(MessageReceived);
  }
  // Go to https://luis.ai and create a new intent,
  // then train/publish your luis app.
  // Finally, replace "GetActivityLocation" with the name of your newly
  // created intent in the following handler.
  [LuisIntent("GetActivityLocation")]
  public async Task GetActivityLocation(IDialogContext context, LuisResult result)
  {
    await context.PostAsync(
      $"You have reached the GetActivityLocation intent. You said:
      {result.Query}"); //
    context.Wait(MessageReceived);
  }

Each time the model determines that the input matches a Get­ActivityLocation intent, it’ll call my bot method. The result is returned by LUIS in the LuisResult object and includes entities such as location and activity. For example, I want to check if the entities contain a country. To do this I use the TryFindEntity method of the LuisResult object, looking for the “builtin.geography.country” type of entity. LUIS provides a number of pre-built entities, which greatly simplifies the job for you. You can find a complete list of these entities at bit.ly/2kWgCHR.

Notice that the builtin.geography entity contains sub-entities, such as country and city. An entity that contains sub-entities is called a composite entity. In this case, I’m interested specifically in the country sub-entity, as shown in Figure 6.

Figure 6 Finding the Country

// Go to https://luis.ai and create a new intent, then train/publish your luis app.
  // Finally, replace "GetActivityLocation" with the name of your
  // newly created intent in the following handler.
  [LuisIntent("GetActivityLocation")]
  public async Task GetActivityLocation(IDialogContext context, LuisResult result)
  {
    await context.PostAsync(
      $"You have reached the GetActivityLocation intent. You said:
      {result.Query}"); //
    EntityRecommendation country;
    if(result.TryFindEntity("builtin.geography.country", out country))
    {
      await context.PostAsync($"Country: {country.Entity}");
    }
    context.Wait(MessageReceived);
  }

Practical Bot Tips

Microsoft is pushing hard on the bot development front, releasing tools like the Language Intelligence Service (LUIS) and Azure Bot Service to help streamline the creation and management of this new class of software. The good news is, most apps today can be easily transformed to bots, with the conversational UI (CUI) a great fit for multiple tasks and types of apps. To get the most out of your bot development effort, here are a few things to consider:

Play for Reach: One of the most compelling things about bots is they run in channels that are found on every PC, tablet and smartphone—like Skype, Facebook Messenger and Telegram. Maximize audience reach by publishing bots to multiple channels. Also, take advantage of available services to localize your bot to multiple languages.

Be a Conversationalist: Make good use of the conversational interface, which is a great fit for multiple tasks and types of apps. For best results, focus on conversational interfaces like chat, speech and language, rather than traditional UI controls like buttons.

Service, Please: The new Azure Bot Service is a cloud-based platform that minimizes infrastructure overhead and allows your bot projects to scale. Behind this capability is Azure Functions, which enables developers to host small pieces of code in the cloud.

Get Focused: When building bots, focus on the specific task or tasks each bot needs to solve. Don’t overcomplicate—more AI isn’t always better. Start with the primary task your bot needs to solve, then inject intelligence.

Speak Up: Make use of the smarts built into LUIS to enable open conversations and provide input for your bot. For best results, take care to train your LUIS model, focusing on a concise set of entities and intents. The less “fuzziness” in your model, the better the AI will perform!

Of course, I also need to know the activity entity (you can use Active Fitness to track more than 50 activities, such as running, cycling, skiing and snowboarding). So, when I ask: “Where’s the best skiing in Austria?” I expect the Active Fitness bot to return “skiing” as my activity entity, as shown in Figure 7.

Figure 7 The GetActivityLocation Intent in action with Bot Emulator

Consequently, I updated my GetActivityLocation method to return the activity. Notice that unlike builtin.geography.country, my activity is not a built-in entity. Still, LUIS figures it out pretty well once I provide a few utterances while training the model, as shown in Figure 8.

Figure 8 Training the Model with Utterances

// Go to https://luis.ai and create a new intent, then train/publish your luis app.
  // Finally, replace "GetActivityLocation" with the name of your newly
  // created intent in the following handler.
  [LuisIntent("GetActivityLocation")]
  public async Task GetActivityLocation(IDialogContext context, LuisResult result)
  {
    await context.PostAsync(
      $"You have reached the GetActivityLocation intent. You said:
      {result.Query}"); //
    EntityRecommendation country;
    if(result.TryFindEntity("builtin.geography.country", out country))
    {
      await context.PostAsync($"Country: {country.Entity}");
    }
    EntityRecommendation activity;
    if (result.TryFindEntity("activity", out activity))
    {
      await context.PostAsync($"Activity: {activity.Entity}");
    }
    context.Wait(MessageReceived);
  }

Bot Framework provides in-depth generic code examples for C# at bit.ly/2gHupjg, and for Node.js at bit.ly/2kWolWx.

Get Smart About Location

Many bots provide location-based responses, but coding around location responses can get tedious. In the previous example, I used LUIS intelligence to get a built-in geography entity parsed by LUIS from the conversation. Fortunately, the Microsoft Bing location control is now part of Bot Framework, which makes gathering location-based data much easier for developers.

Location control for Bot Framework also provides a visual interface that includes maps (see Figure 9). This can be very convenient if your bot needs to provide addresses that include ZIP codes, city, region and locality data from an easy-to-use or native interface that works in channels such as Skype and Facebook Messenger.

Figure 9 Location Control Using the Bing Maps Visual Interface

To start using a Bing Location control, simply obtain a Bing Maps API key and a Location dialog component for either a .NET project (via NuGet) or Node.js (via npm), then initialize and call an instance of the Location dialog in your code:

var options = LocationOptions.UseNativeControl | LocationOptions.ReverseGeocode;
var requiredFields = LocationRequiredFields.Locality |
                     LocationRequiredFields.Region |
                     LocationRequiredFields.Country;
var prompt = "Where are you looking for trails?";
var locationDialog = new LocationDialog(
  apiKey, this.channelId, prompt, options, requiredFields);
context.Call(locationDialog, this.ResumeAfterLocationDialogAsync);

LocationDialog provides options for requesting various address fields and customizing prompts. When the result is returned, you can handle it in a resume method, as shown in Figure 10.

Figure 10 Handling Location Results in a Resume Method

private async Task ResumeAfterLocationDialogAsync(
  IDialogContext context, IAwaitable<Place> result)
  {
    var place = await result;
    if (place != null)
    {
      var address = place.GetPostalAddress();
      var formattedAddress = string.Join(", ", new[]
      {
         address.Locality,
         address.Region,
         address.Country
      }.Where(x => !string.IsNullOrEmpty(x)));
      await context.PostAsync(
        "Where are you looking for trails " + formattedAddress);
    }
    context.Done<string>(null);
  }

Beyond LUIS

In addition to LUIS, Cognitive Services provides a number of other APIs that can empower your bots with skills that may seem incomprehensibly complex for an individual developer or business to implement. Luckily, you don’t have to do it on your own:

• Recommendations API allows you to include recommendations on products (such as products frequently bought together) or personalized user recommendations.
• Vision API adds advanced image and video skills to your bots to recognize objects, people’s faces, age, gender or even feelings.
• Academic Knowledge API can add academic knowledge, create Q&As, and add specific knowledgebase skills to your bot.

It’s worth mentioning that bots are also open to third-party APIs and services. You can use your own services and APIs to make your bots unique and solve problems that haven’t been solved before. Indeed, Bot Framework brings a new world of opportunities, as well as human knowledge evolution and accessibility, via simple conversational interfaces and easy-to-use APIs.

New and Exciting

Besides regular updates to Bot Framework for the .NET Framework and Node.js, as well as REST APIs, Microsoft recently added features that take bot development to the next level.

One of the latest additions to Bot Framework, Azure Bot Service, now offers an easy way to leverage the convenience and scalability of the cloud for your bot projects (bit.ly/2knEtU6). Bot Service, shown in Figure 11, uses Azure Functions and a collection of quick-start templates to help you get the most out of your bot code and allow your bots to scale to any level (bit.ly/2kuo9Bb). For example, you can choose from Basic, Form, Proactive, LUIS, or Question and Answer bot templates, each providing a ready-to-run bot you can expand as needed.

Figure 11 Editing Bot Service Code Directly in the Browser

Azure Functions is the technology behind Azure Bot Service, providing event-driven experiences based on serverless architecture and compute-on-demand. What this really means is that you no longer need an expensive app-hosting infrastructure for each bot. Instead, you can make bots much more lightweight, scale them more easily and save on development effort and time, all while simplifying connectivity with Cognitive Services.

QnA Maker (qnamaker.ai) is another addition worth mentioning. Available from Azure Bot Service as a template bot project, QnA Maker fills a very common scenario for bots that answer typical questions from an existing knowledgebase.

Wrapping Up

Microsoft Bot Framework offers a quick and easy way to start building bots, providing a conversational interface for new and existing apps or services. It lets you create new experiences and reach billions of users connected through Skype, Facebook Messenger, Telegram and other channels. Microsoft Cognitive Services includes many intelligent APIs you can add to your bot.

The Bot Framework ecosystem is growing rapidly, and you can take advantage of this by adding services included with Cognitive Services or using APIs from third-party providers to make your bots smarter and more capable. The journey that begins with a very simple bot can lead to many discoveries for you and your users. Bots are indeed the new apps (or extensions of existing apps), and with intelligence added, they truly become very powerful. Moreover, they reduce your development time, effort and cost.

Meet My Bots

I’ve developed quite a few bots over the last few months, so I decided to share them with you from one convenient location, listed in Figure A. Check them out and see if they might inspire you to create your own. (The link provided with each adds that bot to Skype.)

Figure A My Collection of Bots

Bot	Description
Solitaire (bit.ly/2jrzP7S)	Solitaire is considered the world’s most popular card game. Now instead of launching an app, you can use your favorite chat client, like Skype, to enjoy the game.
Active Fitness (bit.ly/2knZVbr)	Find trails around the world—running, cycling, walking or skiing–all from world’s top fitness social network.
UNO (bit.ly/2k4AzyH)	Play this world-famous card game on Skype, Messenger or Telegram.
Freecell (bit.ly/2l0NM9b)	Freecell is a great and challenging card game with multiple levels of difficulty. Start with easy levels and learn to play.
Crazy Eights (bit.ly/2kY7EdN)	Check out this easy-to-learn card game.
Card Games Chest (bit.ly/2klXOCV)	Learn to master some of the most popular card games, including Solitaire, UNO, Crazy Eights, 101, Freecell and Mau-Mau.

---

Kevin Ashley (@kashleytwit) is an architect evangelist for Microsoft. He’s coauthor of “Professional Windows 8 Programming” (Wrox, 2012) and a developer of top apps, bots and games, most notably Active Fitness (activefitness.co). He often presents on technology at various events, industry shows and Webcasts. In his role, he works with startups and partners, advising on software design, business and technology strategy, architecture, and development. Follow his blog at kevinashley.com and on Twitter: @kashleytwit.

Thanks to the following Microsoft technical expert for reviewing this article: Mat Velloso

---

Discuss this article in the MSDN Magazine forum