Geoff Innis

Lightweight Geospatial in Dynamics 365 for Phones

Geoff Innis — Wed, 13 Mar 2019 17:05:45 GMT

Bringing geospatial insights to business solutions is certainly nothing new; mapping and spatial searching are core components of the Dynamics 365 for Field Service, and many ISV solutions are available to glean location-based insights to optimize operations in Dynamics. In this post, we will see how we can quickly and easily bring lightweight geospatial capabilities to the Dynamics 365 for Phones application, with nothing more than a Bing Maps key, and a Web Resource that we will create. This 60 second video (with voice-over) shows our end-result in action: [video width="1920" height="1080" mp4="https://msdnshared.blob.core.windows.net/media/2019/03/MapsOnMobile.mp4"][/video] We will enable our users in the field to:

Quickly open account records based on proximity, saving time and increasing adoption
Visualize customer buildings and sites to streamline visits
Identify nearby accounts for ad-hoc visits, to optimize time in the field

We will create an HTML Web Resource that will identify the user’s location, search the Web API to find nearby accounts, then plot those accounts (with corresponding entity images) on a map. While in the field, users can launch their app, and immediately bring up account records as they approach. Prerequisites The prerequisites for building and deploying our account mapping in D365 for Phones include:

An instance of Dynamics 365 (Online)
- You can request a trial of Dynamics 365 here
Geocoded Account Records, with address1_latitude and address1_longitude attributes populated (note that the Field Service module for Dynamics includes in-built geocoding capabilities)
The Dynamics 365 for Phones app installed on your mobile device
The text editor of your choice
A Bing Maps for Enterprise key

You can obtain a key using the instructions found here

Creating our HTML Web Resource In a Web Resource, we can use JavaScript to not only access the Dynamics Client API, but also the Bing Maps v8 control for interactive mapping. As a result, most of our work will be done in JavaScript. Using the text editor of our choice, we will begin building our web resource. We start with our HTML document, in which we include the JavaScript library for our Dynamics Client API, along with some global variables defined in JavaScript: [snippet slug=maps-on-mobile-web-resource-start line_numbers=false lang=html] Creating our Map We define our GetMap function (to be called when the Bing Maps JavaScript library has loaded) such that it will:

find the user’s location using the Xrm.Device.getCurrentPosition method
instantiate an interactive map in our HTML page, and
call the getNearbyAccounts function, passing the user's location

Finding Nearby Accounts

getNearbyAccounts

build and execute a retrieveMultipleRecords call against the D365 Web API using a bounding box around our user’s location
load the Microsoft.Maps.SpatialMath module to enable us to readily determine distances between locations
sort our locations by distance, using the compareDistance function
call our AddPushpinCanvas function for each account
set the map view to optimally display all accounts and the user’s location

Adding Pushpins with Account Images and Click Handlers

AddPushpinCanvas

define a function to call after our account entity image has been loaded, to:
- create a Canvas element
- add the entity image to the canvas, cropped to a circle
- add a border to the canvas
- create a Pushpin using the Canvas we created as the icon
- add a click handler to the pushpin, to open the appropriate account record
- call the specified callback function, which will then add the pushpin to the map
load the entity image for the account, subsequently triggering the onload function

Our HTML Web Resource Body

a DIV to present messages to the user as an overlay
a DIV to host our interactive map
a SCRIPT tag to load the Bing Maps v8 map control, with our GetMap function specified as the function to call upon load

Creating a Dashboard

Settings > Customizations > Customize the System > Web Resources > New

Note:

Enable Mobile

Settings > Mobile Settings

User Content and Location

here

Phone Call Transcription and Insights with Video Indexer and Dynamics 365 (Part 2)

Geoff Innis — Thu, 30 Aug 2018 15:04:15 GMT

This post is the second in a series in which we explore how we can leverage Microsoft’s Cognitive Services to greatly enhance the insights we can derive from phone call recordings in an automated way. We can leverage the investments Microsoft has been making in machine learning and artificial intelligence to automatically extract metadata from audio and video files, including conversation keywords, a sentiment timeline, a call transcript, and more. In Part 2, we will build upon the Custom Actions and Workflow that we built in Part 1 by bringing in rich Insights, Transcript, and Call Audio widgets directly into our Phone Call record in Dynamics, with the ability to view our full transcript, visualize our sentiment timeline, and search the call, with an integrated media player allowing us to jump to and listen to key moments in the call: Insights Widget, including interactivity with Audio Widget Transcript Widget, including interactivity with Audio Widget In order to embed the Video Indexer Cognitive Insights and Player widgets into our Phone Call form in Dynamics, we need to first obtain a Video Access Token from the Video Indexer API, and use that token in our widget URLs. As part of our earlier post, we already have a Custom Workflow Activity that will retrieve a Video Access Token for us. In order to be able to access it from our Web Resource, we will create an Action that is callable from the Dynamics Web API. Custom Action to Retrieve Token Logged in to the Dynamics 365 web client with our administrator credentials, we navigate to Settings > Customizations > Customize the System. We choose Processes from the left navigation, and choose New. We specify that we are creating an Action type of process that is not applicable to a specific entity: Our Input and Output arguments for the Action will be:

Input
- VideoId (String)
Output
- VideoAccessToken (String)
- VideoIndexerAccountId (String)

Our Action will include the following logic:

we call our GetVideoToken Custom Workflow Activity
we check to ensure that our activity completed successfully, and that we have values for our Account ID and Access Token, and if so, we assign values to our VideoAccessToken and VideoIndexerAccountId output arguments

We set the input parameters to the GetVideoToken activity like so: After we Save and Activate our process, we are now ready to build our Web Resource that will call it. Phone Call Insights Web Resource We now create an HTML Web Resource that will be embedded in our Phone Call form in Dynamics, and will present rich and interactive Insights, Transcript and Call Audio widgets. Our Web Resource code contains:

two Iframes to hold the Cognitive Insights Widget and Player Widget
an included Mediator JavaScript file, to manage the interactions between the two widgets
our included ClientGlobalContext JavaScript file to aid our Dynamics client scripting
JavaScript code that will execute on the onload event of our document body, and:
- Retrieve the Video Indexer ID from our Phone Call record
- Build and Execute a request to the Dynamics Web API to call our newly-created Action to obtain a Video Token, via the XMLHttpRequest object
- Obtain the Video Token and Video Indexer Account ID from the response
- Build URLs for the Insights Widget and Player widget using the account ID, video ID, and token
- Use the built URLs to set the Source for the two widget Iframes

[snippet slug=phonecallinsightswebresource line_numbers=false lang=html] Logged in to the Dynamics 365 web client with our administrator credentials, we navigate to Settings > Customizations > Customize the System. We choose Web Resources from the left navigation, and choose New, and we can now provide a name, upload our code, and Save and Publish. Updating our Phone Call Form We now add our newly-created Web Resource to our desired Phone Call form. Logged in to the Dynamics 365 web client with our administrator credentials, we navigate to Settings > Customizations > Customize the System. We choose Entities > Phone Call > Forms from the left navigation, and select our Phone Call for Interactive experience form. To the form, we add a new Tab, and within that Tab, we add our Web Resource: We save our form, publish our customizations, and are now ready to view our Phone Call Insights. We can navigate to our Phone Call record from Part 1, and we can now open and view our interactive Insights, Transcript and Player widgets: Insights Widget, including interactivity with Audio Widget Transcript Widget, including interactivity with Audio Widget We can now:

View our call sentiment timeline, and jump to any point in the call audio
View call keywords, and jump to any instances of those keywords in the call audio
View and Search our transcript, playing audio from any point in the transcript or search results
Translate our transcript into a variety of languages

By deriving call recording insights from the Video Indexer service and bringing them into Dynamics, we can now have a greater understanding of key support issues, customer sentiment, and call quality, not only for individual customers and calls, but across our organization. We can leverage this to help us optimize our support operations, and customer interactions on an ongoing basis.

Phone Call Transcription and Insights with Video Indexer and Dynamics 365 (Part 1)

Geoff Innis — Tue, 10 Jul 2018 09:29:12 GMT

In many contact centers, the telephone continues to be one of the primary channels of engagement for customers. Frequently, the only insights we will have about the nature of a call that has taken place will be based on the typed notes or other data manually entered by the contact center agent. By leveraging Microsoft’s Cognitive Services, we can greatly enhance the insights we can derive from phone call recordings in an automated way. We can leverage the investments Microsoft has been making in machine learning and artificial intelligence to automatically extract metadata from audio and video files, including conversation keywords, a sentiment timeline, a call transcript, and more. This post will be the first in a series in which we explore how we can leverage the new Video Indexer service (currently in preview), along with the robust workflow capabilities of Dynamics 365 for Customer Engagement, to automate the indexing of phone call recordings, augment our Phone Call records with metadata about the contents of the call, and provide users with an interactive experience within Dynamics to gain more insights into the conversation. This first post will focus on creating Custom Workflow Activities that interact with the Video Indexer API to upload and index call recordings, then using those activities within a Workflow process to automatically update our Phone Call record with insightful metadata. In this post, we will build on the sample code available in these two resources:

Create a custom workflow activity in the Dynamics 365 documentation, and the associated code samples in the D365 SDK
The Video Indexer How-to Guide for Uploading and Indexing Videos in the Cognitive Services documentation

By the end of our first post, we will be able to view some derived insight metadata about our recorded phone call in Dynamics: And by the end of our series, we will also be able to leverage a rich Insights and Transcript viewer, with the ability to view our full transcript, visualize our sentiment timeline, and search the call, with an integrated media player allowing us to jump to and listen to key moments in the call: Insights Widget, including interactivity with Audio Widget Transcript Widget, including interactivity with Audio Widget Notes:

In this post, we will assume that we have access to a recording of the telephone call, and that the Phone Call record in Dynamics 365 will be updated with a URL where the audio recording can be retrieved. Many telephony providers offer call recording capabilities, and this data can typically be captured through basic CTI integration.
The Video Indexer service brings together a number of cognitive capabilities into a readily-consumable service, minimizing development. We will use the Video Indexer service in this post, but for those seeking greater control over the underlying cognitive capabilities being leveraged, Azure Media Services can enable developers to achieve similar functionality.

Prerequisites The prerequisites for building and deploying our custom workflow activity include:

An instance of Dynamics 365 for Customer Service (Online)
- You can request a trial of Dynamics 365 for Customer Service here
Visual Studio 2015 or higher, with Windows Workflow Foundation installed
Microsoft .NET Framework 4.5.2
The Dynamics 365 Developer Tools
A subscription for the Cognitive Services Video Indexer API
- As of the time of writing, you can obtain a free subscription to this preview service API following the instructions found here.
- Follow the instructions in the above link to obtain your Account ID, Primary Key and Location parameter (‘trial’ for trial accounts, but may differ if you connect your account to Azure)

Adding Custom Fields to Phone Call Entity Before we create our workflow to obtain insights about phone call recordings, we will add some custom fields to the Phone Call entity. Many of these will be updated with insights from the Video Indexer service. These fields include:

Call Recording URL (Single Line of Text): to store the URL where our call recording is stored; it is assumed this will be programmatically set by the telephony provider
Video Indexer ID (Single Line of Text): to store the ID of our call index in the Video Indexer service
Call Keywords (Multiple Lines of Text): to store a comma-separated list of keywords from the Video Indexer insights
Negative Sentiment Ratio (Decimal Number, minimum 0, maximum 100): to store a value representing the percentage of the call recording in which negative sentiment was detected
Positive Sentiment Ratio (Decimal Number, minimum 0, maximum 100): to store a value representing the percentage of the call recording in which positive sentiment was detected
Neutral Sentiment Ratio (Decimal Number, minimum 0, maximum 100): to store a value representing the percentage of the call recording in which neutral sentiment was detected

For a detailed walkthrough of adding new fields to D365 for Customer Engagement, see here. Building our Custom Workflow Activities In Visual Studio, we will first create a new project by selecting Workflow under Visual C# in the Installed Templates pane, and then select Activity Library. We will name our project CallAudioAnalysis. In our Project Properties, on the Application tab, we specify .NET Framework 4.5.2 as the target framework. Adding References We add references to our project by right-clicking the CallAudioAnalysis project in the Solution Explorer, and adding the following:

Microsoft.Xrm.Sdk
Microsoft.Xrm.Sdk.Workflow
System.Net
System.Net.Http
System.Runtime.Serialization

Note that the Microsoft.Xrm.Sdk and Microsoft.Xrm.Sdk.Workflow assemblies can be retrieved via NuGet, by obtaining the following packages using the NuGet Package Manager Console:

https://www.nuget.org/packages/Microsoft.CrmSdk.Workflow
https://www.nuget.org/packages/Microsoft.CrmSdk.CoreAssemblies

Adding a Data Contract for the Video Indexer API To facilitate interacting with the Video Indexer Service, we add a new item to our project: a Visual C# class which we will name VideoIndexer.JSON.cs. We will populate this file with JSON data contracts for the data that comes back in the responses for the Upload Video operation, and the Get Video Index operation. Data Contract Code Excerpt Due to Length; for full Data Contract, click here [snippet slug=video-indexer-json-data-contract line_numbers=false lang=c-sharp height=500] Adding Our C# Code - UploadVideo.cs We can delete the Activity1.xaml file in the project, and Add a new Class to the project, which we name UploadVideo.cs. This custom action will be responsible for uploading our call recording to the service. To our new class, we:

add some using statements
make the class inherit from the CodeActivity class and give it a public access modifier
add functionality to the class by adding an Execute method

We define our input and output parameters for our custom activity:

viAccountId as an input parameter: this is the Account ID of our Video Indexer API account, retrieved in our prerequisites
viApiKey as an input parameter: this is the Primary Key for our Video Indexer API account
viApiLocationString as an input parameter: this is the Location parameter for our Video Indexer API account
videoUrl as an input parameter: this is the publicly available URL from which the audio file can be retrieved
videoId as an output parameter: this is the ID of the media, after it has been indexed by the service
processStatus as an output parameter: this is an indicator of the success of the video upload

In our Execute method, we:

Create our Tracing service for debugging, and our Context
Obtain our input parameters
Prepare and issue an HTTP request to the Video Indexer Authorization API, with our API Key, to obtain an ‘Account Level’ Access Token
Prepare and issue an HTTP request to the Video Indexer Operations API, with our Access Token, to upload our audio file
Read the response, serialize it into an UploadVideoResponse object, and retrieve the ID of our uploaded media
Set our processStatus and videoId output parameters

[snippet slug=uploadvideo-cs-custom-action line_numbers=false lang=c-sharp] Adding to Our C# Code - GetVideoIndex.cs Just as with our previous custom action, we Add another new Class to the project, which we name GetVideoIndex.cs. This custom action will be responsible for retrieving the insights from our call recording once it has been indexed by the Video Indexer service. Just as before, to our new class, we:

add some using statements
make the class inherit from the CodeActivity class and give it a public access modifier
add functionality to the class by adding an Execute method

We define our input and output parameters for our custom activity:

viAccountId as an input parameter: this is the Account ID of our Video Indexer API account, retrieved in our prerequisites
viApiKey as an input parameter: this is the Primary Key for our Video Indexer API account
viApiLocationString as an input parameter: this is the Location parameter for our Video Indexer API account
videoId as an input parameter: this is the ID of the media that we are retrieving insights for
keywords as an output parameter: a comma-separated list of keywords from the Video Indexer insights
negativeSentimentRatio as an output parameter: a value representing the percentage of the call recording in which negative sentiment was detected in the insights
positiveSentimentRatio as an output parameter: a value representing the percentage of the call recording in which positive sentiment was detected in the insights
neutralSentimentRatio as an output parameter: a value representing the percentage of the call recording in which neutral sentiment was detected in the insights
durationInMinutes as an output parameter: the duration of the call recording, in minutes
processStatus as an output parameter: this is an indicator of the success of the video upload

In our Execute method, we:

Create our Tracing service for debugging, and our Context
Obtain our input parameters
Prepare and issue an HTTP request to the Video Indexer Authorization API, with our API Key, to obtain a ‘Video Level’ Access Token
Prepare and issue an HTTP request to the Video Indexer Operations API, with our Access Token, to retrieve the indexed insights about our phone call recording
Read the response, serialize it into a GetVideoIndexResponse object, and retrieve the various insight components from our phone call
Set our various output parameters

[snippet slug=getvideoindex-cs-custom-action line_numbers=false lang=c-sharp] Adding to Our C# Code - GetVideoToken.cs Just as with our previous custom actions, we Add another new Class to the project, which we name GetVideoToken.cs. This custom action will be responsible for simply retrieving a Video Access Token. In our next post, we will use this custom action to help us embed further insights into Dynamics 365. Just as before, to our new class, we:

add some using statements
make the class inherit from the CodeActivity class and give it a public access modifier
add functionality to the class by adding an Execute method

We define our input and output parameters for our custom activity:

viAccountId as an input parameter: this is the Account ID of our Video Indexer API account, retrieved in our prerequisites
viApiKey as an input parameter: this is the Primary Key for our Video Indexer API account
viApiLocationString as an input parameter: this is the Location parameter for our Video Indexer API account
videoId as an input parameter: this is the ID of the media that we are retrieving insights for
videoAccessToken as an output parameter: this is the Video Access Token, valid for 1 hour
viAccountIdOut as an output parameter: this is the same Account ID, output for use in conjunction with the Access Token
processStatus as an output parameter: this is an indicator of the success of the video upload

In our Execute method, we:

Create our Tracing service for debugging, and our Context
Obtain our input parameters
Prepare and issue an HTTP request to the Video Indexer Authorization API, with our API Key, to obtain a ‘Video Level’ Access Token
Set our various output parameters

[snippet slug=getvideotoken-cs line_numbers=false lang=c-sharp] Before compiling our assembly, we sign it. In the project properties, under the Signing tab, we select Sign the assembly and provide a key file name. We are now ready to compile the assembly by Building the solution. Registering our Assembly We now need to register our custom workflow activity assembly on our Dynamics 365 Online instance. To do that, we will use the Plug-in Registration Tool. This tool is available in the Dynamics 365 Developer Tools. Following the instructions in the documentation, we launch the tool, and authenticate using our administrator credentials for D365. We then select Register New Assembly from the Register menu. In the resulting dialog box, we choose the location of our compiled assembly (which should be in the CallAudioAnalysis\bin\Debug folder). We select our assembly and our workflow activity for registration. We specify Sandbox as the isolation mode, and Database as the storage location. Finally, we click Register Selected Plugins: We now have a custom workflow activity that will allow us to interact with the Video Indexer API to derive insights from call recordings. Creating our Workflow We will now create a workflow that will make use of two of our custom actions, to upload our call recording to the service, wait until it has completed processing, then retrieve the indexed insights and update our Phone Call record. Logged in to the Dynamics 365 web client with our administrator credentials, we navigate to Settings > Customizations > Customize the System. We choose Processes from the left navigation, and choose New. We specify that we are creating a Workflow type of process that is applicable to the Phone Call entity, running in the background, and started when a record is created, or when the Call Recording URL field is updated: Our workflow will include the following logic:

we check to ensure we have a Call Recording URL, and if so:
- we use our UploadVideo custom action to upload our video
we check to ensure our request to upload the video was successful, and if so:

we update our Phone Call record with the ID from the Video Indexer Service
we wait a nominal duration of 1 hour for our audio to complete processing; this logic could be extended with recursive requests until indexing was complete, to expedite retrieving of the insights, and to avoid problems if processing takes longer than 1 hour
after our wait time, we use our GetVideoIndex custom action to retrieve our insights

we check to ensure our request for the insights was successful, and if so:
- we update the phone call record with the insights

The complete workflow logic looks like this: We set the input parameters to the UploadVideo action like so: We set the input parameters to the GetVideoIndex action like so: We update our phone call with insights like so: After we Save and Activate our process, we are now ready to test it. We can test the indexing process by creating a Phone Call record and setting the Call Recording URL attribute to a value containing the URL to a call recording. In this example, we are using a .M4A audio file, hosted in Azure Blob Storage. The recording contains a simulated phone call regarding a problem connecting an Xbox One to a network. We can track the progress of our custom actions using the Plug-in Trace Logging feature of Dynamics. After one hour, we should be able to view insights about our phone call in Dynamics 365. This screenshot shows an updated Phone Call form, containing our newly-created custom fields: Note that the Video Indexer service can provide a wealth of additional insights about our phone call, including the full transcript, the language being spoken, brands mentioned, instances of banned words, and much more. The custom activities and the workflow can be further extended to bring this additional metadata into Dynamics 365, if desired. In our next post, we will build upon this one, bringing interactive Insights, Transcript, and Call Audio widgets directly into our Phone Call record in Dynamics.

Enhanced Knowledge Base Usage Analytics with Azure Application Insights and Power BI

Geoff Innis — Fri, 09 Mar 2018 14:15:28 GMT

When managing a self-service knowledge base, understanding how customers and employees are searching for and consuming self-service knowledge is an important part of ensuring your content remains relevant and useful for end users. Dynamics includes a number of in-built capabilities to help analyze the usage of the knowledge base, including:

Tracking of article views by source
Tracking of article ratings and feedback
Tracking of case deflections from knowledge
Tracking of articles associated with, or used in resolving cases

In some instances, we may wish to extend beyond the in-built capabilities, to understand usage patterns to a greater depth: what your users are searching for, which searches are not returning any results, and how users are navigating through your Dynamics portal, for example. These metrics can help you create targeted content to meet the needs of your users. In this post, we will walk through an example of how we can track and analyze portal-based knowledge base usage patterns in more detail, with Azure Application Insights and Power BI. We will augment the in-built analytics with reporting on:

Search patterns – including top searches and top failed searches
Article View patterns – including views by user and views by referring page

Our goal will be to empower knowledge managers with a visualization such as this: Prerequisites The prerequisites for building and deploying our enhanced analytics include:

An instance of Dynamics 365 for Customer Service (Online)
- You can request a trial of Dynamics 365 for Customer Service here
A Dynamics 365 Portal connected to your Dynamics 365 instance

In this walkthrough, we will use a portal configured with Portal Audience: Customer, and type of portal as Customer Self-service Portal

A Microsoft Azure subscription for enabling our Application Insights resource
Power BI Desktop, and a Power BI account, which can be obtained here

Setting Up Application Insights Azure Application Insights is an extensible Application Performance Management (APM) service that works across multiple platforms. Among its capabilities are the ability to track and understand what users are doing with web applications, to continually improve performance and usability. We will use it to track and report on specific user actions within the Dynamics 365 portal, leveraging Application Insights for web pages, and the Application Insights SDK JavaScript API. First, we authenticate to the Azure Portal, and create a new Application Insights resource. This link will take us directly to the Application Insights resource creation blade. We complete the required information, specifying the Application Type as ASP.NET Web Application, and click the Create button: Once the Application Insights resource has finished deploying, navigate to the resource. Scroll down and click the Getting Started button in the resource blade, click MONITOR AND DIAGNOSE CLIENT SIDE APPLICATION, then click to copy the JavaScript code snippet to your clipboard: In the Dynamics 365 Web Client, we can paste this code into a Content Snippet named Tracking Code, which is suited for this purpose. The code we are pasting includes our Application Insights key. This code will be included in each page that is rendered on our portal, enabling us to track portal usage: We will now leverage the Application Insights SDK JavaScript API to track specific events that occur on the portal. Within the JavaScript SDK, we are able to make use of the trackEvent method to log instances of specific user actions, including our own properties that we can specify. We will use this to log user searches, and knowledgebase article page views. We can also ensure that our events are tracked to the specific portal user by setAuthenticatedUserContext method to associate searches and views to individual authenticated portal users. Tracking Searches When users search on our portal, we will track what query the user entered when searching, as well as how many results were returned. If we have an authenticated portal user, we will also include that in the context of our event. We do this by editing the default Web Template called Faceted Search – Results Template, which is used to render the results of the search on the portal. The portal leverages Handlebars, the popular JavaScript templating engine, to render the results. The updated template code below shows demo-grade updates which will register a Handlebars helper JavaScript function called logresults, which can be called from within the Handlebars template, where we have access to the count of search results that are being rendered. The helper will:

Receive the result count as a parameter (note that we are tracking the number of results across all types; articles, forums, portal pages, etc.)
Ensure that the count is a numeric value
Set the authenticated user’s GUID into context, if we have an authenticated user (making use of Liquid templating to inject the authenticated user’s ID)
Log a custom event named search, with the results count, and the query phrase (also injected vi Liquid) included as custom properties
Use a counter to ensure that the search event is logged only once per page render, and not logged again on subsequent uses of the search faceting

[snippet slug=faceted-search-results-template line_numbers=false lang=js] Tracking Article Views Although Dynamics 365 portals track article views as an in-built capability, we can gain more insight into usage patterns by logging custom events. We can track page views by specific users, track how they arrived at the page (by search vs navigation), and more. To do this, we will navigate to the portal Web Page called Knowledge Base – Article Details (drill down into the Localized Content for the page, if applicable) and we add some Custom JavaScript to the page, found under the Advanced tab in the default Web Page form. The following demo-grade JavaScript code will:

Use a JavaScript regular expression to obtain the Article ID from the page URL (ensure that the pattern used matches your article public number convention)
Checks the referrer URL against a number of the Site Settings pages that we anticipate article traffic to come from, and setting a source variable accordingly
Sets the user’s GUID into event context, if the user is authenticated
Logs a custom event named view, with the articleId and source as parameters

[snippet slug=knowledge-base-article-details-custom-javascript line_numbers=false lang=js] After saving these changes, we can open up our portal in a browser, and start searching and viewing knowledgebase articles, to generate some data for our reporting and analytics. Viewing and Exporting the Event Data in Application Insights Back in the Azure portal, on our Application Insights resource blade, we navigate to the Overview, which allows us to click the Analytics button to open Analytics: Analytics is the search and query tool that accompanies Application Insights. It has its own query language, which can be used to run queries against the data. Queries we write can also be used to export data to Power BI. In Analytics, we open a new tab to query our data, and type in a query that will return all of our search events. Note how we retrieve the query term and result count from our custom parameters: Note that we also retrieve all data from the last 90 days. Application Insights stores analytics data for up to 90 days. If we wish to report on usage patterns going back further than 90 days, we could make use of the Continuous Export feature of Application Insights to export the data to a storage account, and into SQL Azure for continued availability in our analytics reports (not covered in this post). We will now export the query for use in Power BI, by selecting Export to Power BI from the Export menu: We will be prompted to open or save the text-based exported query. We can open and then copy the query to our clipboard, for use in Power BI: We then open Power BI Desktop, and click the Get Data button, and choose Blank Query: We insert our search query, and click the Done button: We can repeat these steps for our article view query, using the following query syntax: customEvents | extend source = customDimensions.src, id = customDimensions.id | where timestamp > ago(90d) and name == "view" | order by timestamp desc Building our Power BI Dashboard We can now start building a dashboard in Power BI to help us visualize and understand knowledgebase usage trends. For example, we can use our Search Query data in a Bar Chart visualization, selecting itemCount and query, with a filter in which resultcount is equal to 0, to create a visualization of top failed searches: We can continue to build out a dashboard, creating visualizations that will help us understand our users’ search and article viewing patterns, using attributes such as authenticated user, timestamp, user location, and more. We can also pull data from Dynamics 365 into Power BI to augment our visualizations further: Finally, we can use the process outlined here to embed our dashboard in Dynamics 365: We can now analyze the patterns of usage in our knowledge base on an ongoing basis, to make informed decisions about how best to augment our knowledge, and how to optimize our end-users' self-service experiences. These same techniques can be extended to track and analyze many other aspects of user behavior on Dynamics 365 portals as well.

Bots as Chat Agents, with Escalation to Live Agents

Geoff Innis — Thu, 19 Oct 2017 09:50:56 GMT

Update, Jul. 2018: With the QnA Maker service now in GA, it is now necessary to pass in an additional parameter to the QnAMakerDialog if using a GA QnA Maker Service: "endpoint hostname"; see this link (scroll to QnAMakerDialog heading, and CSharp code sample for details) Update, Feb. 2018: This post has been updated to reflect the new Bot Channels Registration process in the Azure Portal. Bots offer organizations a means of scaling customer service and customer engagement through natural and intuitive conversational experiences, as we’ve seen in some of our earlier posts. But bots will not always be able to meet all of our customers’ needs during an interaction. Being able to seamlessly transition a customer from a conversation with a bot to an actual live agent can be a key capability in realizing scale while also ensuring high-quality customer engagements. In this post, we will see how we can implement seamless escalations from bots to live agents leveraging the capabilities of Live Assist from Microsoft partner CafeX. This one-minute video (no sound) shows our intended functionality in action: [video width="1920" height="1040" mp4="https://msdnshared.blob.core.windows.net/media/2017/10/QnA_Bot_wEscalation_wCallouts.mp4"][/video] Escalation Options with Live Assist With the latest release of Live Assist for Dynamics 365, we have two options available to us in implementing Bot to Agent escalations:

Create a bot and enable it as an agent in Live Assist, with the ability to transfer/escalate to other agents; this is enabled through the Live Assist Bot Agent Connector, and will enable end users to engage with your bot via the Live Assist end-user interface, just as if engaging via Live Assist chat with a Live Agent
Create a bot and enable it on the Bot Framework channels of your choice, adding the ability for chats to be escalated to Live Assist agents via the Live Assist Escalation SDK; this option will enable end users to engage with your bot on the Bot Framework channels of your choice, though not through the Live Assist end-user chat experience

In this post, we will implement the first option: a bot that is enabled as a Live Assist agent, with the ability to transfer or escalate to other “real” agents in Live Assist. We will create a bot that can:

be accessed via the standard Live Assist end-user chat interface
provide answers to customer questions via the Cognitive Services QnA Maker API
detect the sentiment of all customer input via the Cognitive Services Text Analytics API
initiate a transfer to other agents or skills when negative sentiment is detected

Note that we will be building our bot using the Bot Builder SDK for .NET. If you are working with Node.js, CafeX offers sample code for enabling a Bot Agent that can provide answers via QnA Maker here. Prerequisites The prerequisites for building and deploying our escalation bot include:

An instance of Dynamics 365 for Customer Service (Online)
- You can request a trial of Dynamics 365 for Customer Service here
A Live Assist for Dynamics 365 Trial that has been activated for your Dynamics 365 organization
Optional: you may wish to provision and configure a Dynamics 365 Portal, and embed Live Assist chat in your portal
A Microsoft Azure subscription for hosting our bot, and for enabling our Cognitive Services resources
Visual Studio 2017
Microsoft .NET Framework 4.6.1
The Bot Builder SDK for .NET
The Bot Application, Bot Controller, and Bot Dialog Visual Studio Templates
- Save the downloaded zip file to your Visual Studio 201X templates directory, which is typically found in %USERPROFILE%\Documents\Visual Studio 2017\Templates\ProjectTemplates\Visual C#

Setting Up our QnA Maker Service Microsoft's Cognitive Services QnA Maker provides an easy-to-use tool for the ingestion of "Frequently Asked Questions" (questions and answers), making them available via a service that can be readily consumed within the Bot Framework. Bots can provide answers to end-user questions through the use of a Bot Framework dialog that we will leverage later in our post. The QnA Maker documentation provides a brief video walkthrough of how to access the QnA Maker service on https://qnamaker.ai, and to create and publish a question and answer service. Rather than recreate the documentation, follow along with the steps seen in the video here. While following along with the quick-start video, we obtain our credentials for accessing the knowledge base service we have created, including our:

QnA Maker Subscription Key
The Knowledge Base ID
The Knowledge Endpoint URL

Save these credentials for use later in our setup process. Optional: If you wish to populate your QnA Maker service with the articles from your Dynamics 365 knowledgebase, you can export your articles from Dynamics 365 to Excel, then format the articles in the tab-separated Question, Answer and Source format outlined in the QnA Maker documentation, for upload into your QnA Maker knowledge base. Obtaining our Text Analytics Endpoint and Key To enable our bot to escalate to an agent when the end user expresses negative sentiment, we will leverage the Cognitive Services Text Analytics API, which is able to be provisioned from the Azure Portal with an Azure subscription. This deep link will take you to the Text Analytics API resource creation blade in the Azure Portal. Complete the required details, providing your desired resource name, subscription, location, pricing tier, and resource group (either a new or existing one). Accept the terms, and click the Create button: You can then view details of the Text Analytics API resource that you have created, to obtain your endpoint based on the region you selected, and copy one of the two keys that were created for you: Save the endpoint and key for later use. Important Note: When we insert the endpoint into our app configurations later on in the post, we will append '/sentiment' to the endpoint copied from the Azure portal. More details found below. Building our Bot We will now start putting together the code for our bot, using steps found in the Create a bot with the Bot Builder SDK for .NET quickstart. We Open Visual Studio, and create a new project using the Bot Application template that we downloaded as a part of our prerequisites: In our Project Properties, on the Application tab, we specify .NET Framework 4.6.1 as the target framework. Installing Packages and Adding References We need install several packages that we will leverage as a part of our bot code, and ensure our project is referencing the latest version of the Bot Builder SDK. We can do this via NuGet:

The latest version of the Microsoft.Bot.Builder package
The Microsoft.Bot.Builder.CognitiveServices package, for QnA Maker functionality

Adding a Data Contract for the Text Analytics API To facilitate interacting with the Cognitive Services Text Analytics API, we add a new item to our project: a Visual C# class which we will name TextAnalytics.JSON.cs. We will populate this file with the JSON data contracts for the REST Services outlined in the Text Analytics API documentation. [snippet slug=usdsentiment_datacontract line_numbers=false lang=c-sharp] Adding our QnA Maker Dialog To our project, we will add a Bot Framework dialog to manage the interaction with our QnA Maker knowledge base. We right-click on our Dialogs folder in our project, and choose to Add New Item. We will add a C# Class that we name QnAMakerDialog.cs. We will add code to our class using the Basic QnA Maker template found in the Bot Framework documentation here. The dialog will inherit from the QnAMakerDialog object, and calls the QnA Maker service with the following parameters:

Subscription Key: this key was obtained earlier, when setting up our QnA Maker service; we will populate this from application configuration parameters in a subsequent step
Knowledge Base ID: this ID was also obtained while setting up our service; we will populate this from application configuration parameters in a subsequent step
Default Message: this optional parameter allows us to specify the message to return to the user when no answer has been found with a sufficient match
Score Threshold: this optional value between 0 and 1 allows us to specify the threshold of the match that must be found from our knowledgebase in order to return an answer to the user

We add the following code to our new class, to allow the calling of the QnA Maker Service we created: [snippet slug=qnaescalation_qna-maker-dialog line_numbers=false lang=c-sharp] Adding our Escalation Dialog We will also add a simple dialog that will handle escalation scenarios, by confirming that the user wishes to be transferred to an agent, and executing the transfer. We right-click on our Dialogs folder in our project, and choose to Add New Item. We will add a C# Class that we name Escalation.cs. Our Escalation dialog will initiate with the StartAsync method by requesting confirmation from the user on whether they would like to be transferred to a live agent. The response that the user provides is managed by the AfterEscalationConfirmation method. If the user wishes to be transferred (which we test by looking for a response that begins with a ‘y’), a transfer message is constructed and sent, using the transfer message format as outlined in the Live Assist Bot Agent Connector documentation. The key information about the transfer is included in the ChannelData property of the message, including the skill or agent that the chat should be transferred to. In our code, we are transferring the chat to a Skill that will be configured in Live Assist called BotEscalation. We could also adapt the code to transfer to an agent, by replacing our skill attribute with 'agent':'AgentNickname' in our JObject. Our Escalation.cs code is shown below: [snippet slug=qnaescalation_escalationdialog line_numbers=false lang=c-sharp] Adapting our Root Dialog We will now adapt our RootDialog.cs class to examine incoming messages, and respond appropriately depending on the type of Live Assist message it is, and the sentiment expressed if it is a message from the user. Our Root Dialog will:

Respond with a greeting on initial connection to the user
Check the sentiment of all user messages
Route messages with a sentiment score of less than a nominal value of 0.2 to our escalation dialog
Route messages that don’t have negative sentiment to the QnA Maker dialog

Our Root Dialog class also includes methods to call the Text Analytics API to determine the sentiment of user messages, as used in previous posts leveraging the API via .NET. [snippet slug=qnaescalation_rootdialog line_numbers=false lang=c-sharp] Adapting our Message Controller We also make minor updates to our MessageController.cs class, which controls the flow of incoming user messages. We check to see if our incoming message is on the Direct Line channel (which Live Assist agent bots use), and if so, we direct it to our Root Dialog. Otherwise, we direct it straight to the QnA Maker dialog, so that other channels can still benefit from the QnA Maker knowledge base: [snippet slug=qnaescalation_messagecontroller line_numbers=false lang=c-sharp] Later on we will also update our Web.config file with necessary configurations and credentials, but first we need to deploy our bot to Azure, and register our bot. Deploying our Bot to Azure from Visual Studio We will now deploy our bot to a Microsoft Azure App Service. We right-click on our project in the Visual Studio Solution Explorer, and select Publish: We select Microsoft Azure App Service as the project type and click Publish: Next to App Name, we click Change Type and change the service's type to Web App. We enter our desired app name, Azure subscription, Resource Group (new or existing) and App Service Plan (new or existing), and click the Create button: The creation of the needed resources, and the publishing of our bot to the App Service, should now be initiated. Upon completion, we can obtain the Site URL for our bot endpoint, which we can save for later use: Registering our Bot with the Bot Service We will now register our bot with the Bot Service, basing our process on the documentation found here. First, we authenticate to the Azure Portal, with our Azure credentials. Once authenticated, we create a Bot Channels Registration, by clicking the Create a resource button found on the upper left-hand corner of the Azure portal, then select AI + Cognitive Services > Bot Channels Registration. When creating our Bot Registration, we include:

your bot's Name; this is the unique handle for your bot
the Azure subscription, resource group, location, and pricing tier of our choosing
our bot's Messaging endpoint, based on the Site URL we obtained during the publishing process; Note that we must replace http with https, and we must append /api/messages to our Site URL

We can confirm that the registration has been created by checking the Notifications area in the Azure portal. Upon completion of the registration, we can open the newly created resource, and then click on Settings. We can now provide Bot Profile details for our bot. Taken from the documentation, we can supply information including:

an icon that will represent your bot in the conversation
your bot's Display Name; When users search for this bot, this is the name that will appear in the search results
a Description of your bot; This is the description that will appear in search results, so it should accurately describe what the bot does

We also need to click the Manage link to obtain a Microsoft Application Password. This is the link appearing by the Microsoft App ID: This link will open a window where you can generate a new application password. Click Generate New Password. This will generate a new password for your bot. Save both the App ID and Password for later use: Updating our Web.config Configurations We will now update our project's Web.config file with various authentication and configuration values that we have collected. We add the following appSettings settings:

BotId: our bot’s handle as created during Bot Registration
MicrosoftAppId: our bot’s App ID as obtained during Bot Registration
MicrosoftAppPassword: our bot’s App Password, as obtained during Bot Registration
cogTextAnalyticsSubscriptionKey: our Text Analytics API key, as obtained during the provisioning of our resource in the Azure Portal
cogTextAnalyticsEndpoint: the endpoint for the Text Analytics API, based on the region we selected during resource provisioning in the Azure Portal. Important Note: When we insert the endpoint value into our app configurations, we will append '/sentiment' to the endpoint copied from the Azure portal. For example, if you selected the West US region when provisioning your Text Analytics resource, the full value to insert in this appSetting would be https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/sentiment
QnASubscriptionKey: the QnA Maker subscription key that we obtained while Setting Up our QnA Maker Service
QnAKnowledgebaseId: the QnA Maker knowledge base ID that we obtained while Setting Up our QnA Maker Service

[snippet slug=qnaescalation_webconfig line_numbers=false lang=c-sharp] Once again, we publish our updated bot code to Azure, by right-clicking on our project, and selecting Publish, and then clicking on the Publish button with our same Azure publishing profile. Setting up the Direct Line Connection We will now enable the Direct Line channel for our bot, which is the channel that Live Assist leverages to enable the bot as a Live Assist agent. Still in the Bot Registration resource blade in the Azure Portal, we click on the Channels tab for our bot, and choose to add the Direct Line channel: We then show, copy and save our first Secret key. Leaving all Version options selected, we then click the Done button: Adding our Bot as an Agent in Live Assist CafeX has provided instructional steps for adding a bot as an agent in a quick-start guide, and we will add our bot as an agent based on those steps. First, we log into the Live Assist Administration Portal for the Live Assist-enabled Dynamics 365 organization that we are adding our agent to. The URL for the portal is different for each organization, but you can find your URL either in the “Welcome email” you received upon enabling Live Assist, or from within the Dynamics 365 web client, under Service > Live Assist Administration. We navigate to USERS > Bots, and then click the ADD button: In the resulting form, we enter:

Nickname: our desired name for our bot, as displayed to end users
Licenses: the number of Live Assist licenses to allocate to our bot; for every license, three concurrent chats can be handled by our bot
Connection Type: we leave this as Ms
Connection Key: we enter the Direct Line Secret key that we obtained earlier when configuring the Direct Line channel

We then click the SAVE button to save our bot. Updating Live Assist Skills and Engagement We will now assign appropriate skills to our bot agent and our live escalation agents, along with updating our chat engagement to direct chats to our bot. We open up our Live Assist Engagement Administration URL, which can be found in the Dynamics 365 Web Client, under Service > Live Assist Supervisor. We navigate to USERS, and select our newly created bot: We add a new Skill to our bot user, which we are calling Bot. We then click the Save button: We now select a different “live” agent user who will receive escalations from the bot, and add a Skill which we call BotEscalation. We then Save the updates to our live agent user: We now navigate to the CAMPAIGNS tab, and open up our default campaign (typically named Live Chat on your site). We click the Edit button next to our default engagement: We are brought to the ENGAGEMENT step in the Engagement studio process. We click Next in the bottom-right to take us to the WINDOW step. While on the WINDOW step, we click the Add New button: We provide our new window with a name and description: In our Views dropdown, we enable the Pre-chat survey by clicking the toggle, and then edit the pre-chat survey in the resulting window. We will use the pre-chat survey to determine what type of issue the user wishes to get help on, and direct specific types of issues to our bot agent: We use the Trash icon to delete any existing questions, then add a new question of type Routing. We add our desired question text, and add as many options as desired. For our first option, we add option text, and specify that the option will route to our Bot skill. We can add other options that will route to other skills, including directly to the BotEscalation skill, if desired: We then click the Done button in the Pre-chat survey, and click the Save button in the bottom-right to save our window. With our window now saved and selected, we click the Publish button in the bottom-right of the page, to publish our changes. Testing our Bot We should now be able to chat with our bot agent from any sites that we have enabled Live Assist chat on, or via the Demo Site that CafeX makes available for testing in the Live Assist Administration Portal (to open portal, go to Dynamics 365 Web Client under Service > Live Assist Administration). We will test using the Demo Site. While in the Live Assist Administration Portal, navigate to GET STARTED, use the right chevron to navigate to the Open Live Assist for Dynamics 365 panel, then click the DEMO SITE button: In the resulting sample website, you should see your chat engagement. If you elect to chat, and select the appropriate Routing Question option, you should be able to engage in a conversation with your bot that will be answered via QnA Maker. If sufficiently negative sentiment is expressed by you during the chat, you should be able to escalate to a live agent. Notes: Make sure you are logged into Live Assist with a user that has the appropriate BotEscalation skill. In addition, if you are still authenticated in the Live Assist Engagement Administration portal, you may find that escalated chats will be directed there. You can download the full Visual Studio solution code here.

The Intelligent Agent Assistant: Bots in the Agent Desktop – Part 2

Geoff Innis — Thu, 13 Jul 2017 17:56:21 GMT

This post is the second in a series in which we explore how we can leverage natural, conversational experiences powered by automated intelligence to empower the CSR, by building a contextually-aware Intelligent Agent Assistant, integrated into the Unified Service Desk agent desktop. We will build upon the bot that we brought into the Unified Service Desk and enabled to communicate with the Unified Service Desk by initiating actions within the desktop in Part 1. Part 2 of our series will focus on extending this functionality by:

Proactively provide contextually-relevant information and alerts to the CSR via the integrated bot
Enabling administration of the proactive alert content, buttons, and actions from within the Unified Service Desk configurations

To see a 2-minute video that shows the final result we are building towards, check out Part 1 here. Snapshot of Final Result Extending our Intelligent Agent Assistant Now that we have surfaced our bot in USD using the Direct Line channel, and the WebChat control in a USD Standard Web Application hosted control, and enabled the bot to trigger actions in USD when buttons are clicked through USD’s event moniker syntax, let’s move on to triggering proactive alerts for the CSR in the bot. The alerts will be raised in a three-step process:

An Action Call in USD will use the RunScript Action to inject JavaScript into the Bot Assistant control, including JSON data
JavaScript in the Bot Assistant will use the Backchannel to send an event to the bot with the JSON data
The bot will receive the event, and use the received data to create an alert to the user

We’ll see how we can do this for the example of a "High Churn Risk" warning to our agent, based on negative customer sentiment. Initiating Messages to our Bot with a USD Action Call First, we create an Action Call in USD that will be called whenever we open up a case, initiated from the BrowserDocumentComplete event of the Incident control which we use to load our cases. The Action Call uses the RunScript Action, and passes in a payload of JSON data and other parameters into a postMessage event in the Bot Assistant: We add a condition such that the Action Call only executes if the detected sentiment on the case is below a threshold score of 0.35: The full JavaScript code in our Action Call data is shown here. Note how the JSON data contains all of the information required to construct a rich Thumbnail Card in our bot, including URLs for our buttons that use the event moniker syntax from Part 1 to raise events in USD when clicked: [snippet slug=agentbot2_actioncalljavascript line_numbers=false lang=js] Using the Backchannel to Raise Events with our Bot As outlined in the Backchannel example on Github, by creating a DirectLine object in our page, and then sharing it when creating our WebChat instance, we are then able to use the DirectLine object to post activities such as events to our bot. We now create our postMessage JavaScript function referenced in our Action Call. It receives parameters and uses them to post an activity to our bot via DirectLine, we will have a means of sending information via activities to our bot programmatically. Our postMessage method includes the following parameters:

activityType – the type of activity, such as event, message, typing, etc; in our example, we will be sending activities of type event
activityText – the text property for the activity
activityName – the name of the event we are raising
activityData – the channelData that we will receive in the activity; this will be used to pass important event data that the bot will use
showBotRequest – a parameter that will dictate whether or not we wish to raise an event in USD which will cause the Intelligent Agent Assistant to be displayed in the agent desktop if it is not already; this will be helpful in our alerts to the agent

The function will receive the parameters, use them to post an activity to the bot, and then will trigger a USD event to display the bot, if appropriate: [snippet slug=agentbot2_postmessage line_numbers=false lang=js] Receiving Events in our Bot We now need to enable our bot to receive our events, and respond appropriately. We can do this in our Message Controller class. Typical dialog-based samples for the Bot Framework will send received activities that are Message type to a dialog class, and will have a separate method to handle other types of messages. This is where we can include our code to handle our incoming events. We adapt the HandleSystemMessage method to include logic to handle events. In this method, we:

check to see what the name of the event is; if it is named botMessageAlert, we will:
create a ConnectorClient
create a message reply
call our FormatGenericMessage method, passing in our reply and our JSON data and receive our formatted message as output
use our ConnectorClient to send the message back to our end-user

[snippet slug=agentbot2_messagecontroller line_numbers=false lang=c-sharp] Our FormatGenericMessage receives our reply message and JSON data, and uses the JSON data to create a well-formatted Thumbnail type of rich card: [snippet slug=agentbot2_formatgenericmessage line_numbers=false lang=c-sharp] Now, when we load a case in the Unified Service Desk, our bot will proactively display an alert as shown below, with buttons that will trigger Events and associated Action Calls in USD when clicked: Stay tuned for more, as we cover:

Making our bot aware of contextual information from USD
Enabling our bot for engagement via speech, in addition to typing
and more

The Intelligent Agent Assistant: Bots in the Agent Desktop – Part 1

Geoff Innis — Wed, 28 Jun 2017 06:03:29 GMT

When we think of how bots fit into a Customer Service strategy, the typical scenario envisioned is often that of a customer-facing bot, providing customers a conversational support experience powered by automated intelligence. But even with the deployment of a customer-facing bot, most contact centers will still need to support a significant number of customer engagements with the help of a Customer Service Representative (CSR). This post will be the first in a series in which we explore how we can leverage natural, conversational experiences powered by automated intelligence to empower the CSR, by building a contextually-aware Intelligent Agent Assistant, integrated into the Unified Service Desk agent desktop. By providing a conversational Agent Assistant experience for the CSR that is tightly integrated into the Unified Service Desk, we can:

Expedite agent productivity with fast and intuitive access to insights and procedures
Provide a natural engagement experience
Proactively provide contextually-relevant information and alerts

The Intelligent Agent Assistant that we build will have:

Awareness of the context of the agent desktop and current customer interaction
The ability to provide insights, guidance or alerts based on the current context, both proactively and reactively
The ability to trigger and automate actions and activities within the desktop
The ability to be engaged via voice or text/typing

This 2-minute video shows the types of experiences we will be able to provide for agents at the conclusion of our series (with sound): [video width="1920" height="1040" mp4="https://msdnshared.blob.core.windows.net/media/2017/06/IntelligentAgentAssistant1.mp4"][/video] Building our Intelligent Agent Assistant In Part 1 of our series, we will focus on:

Surfacing a bot within the Unified Service Desk
Enabling the bot to communicate with the Unified Service Desk by initiating actions within the desktop

Prerequisites: The prerequisites for building and deploying our Intelligent Agent Assistant include:

An instance of Dynamics 365 for Customer Service (Online)
- You can request a trial of Dynamics 365 for Customer Service here
A Microsoft Azure subscription for hosting your bot
The Dynamics 365 Software Development Kit (SDK)
Visual Studio 2015 or higher
Microsoft .NET Framework 4.6
The Bot Builder SDK for .NET
The Bot Framework Emulator (for Windows)
The Bot Connector Visual Studio Project Template
- Save the downloaded zip file to your Visual Studio 201X templates directory, which is typically found in %USERPROFILE%\Documents\Visual Studio 201X\Templates\ProjectTemplates\Visual C#

Surfacing a Bot in USD There are several options for incorporating a bot into the Unified Service Desk, including:

Configuring a bot for the Web Chat channel, and hosting it in a USD Standard Web Application hosted control
Configuring a bot for the Direct Line channel, and using the WebChat control for your bot client, hosting your control in a USD Standard Web Application hosted control
Configuring a bot for the Direct Line channel, and creating your own bot client, hosted in a Custom Hosted Control in USD

For our Intelligent Agent Assistant, we will configure a bot for the Direct Line channel, and use the WebChat control for the bot client, hosting the control in a USD Standard Web Application hosted control. This option allows us to leverage a pre-built bot client for our user interface, while still allowing us to programmatically communicate with the bot via the Direct Line REST API, sending and receiving event activities, as outlined in the Backchannel example on GitHub. While we will get into more depth on the specifics of our bot code later on you can review some of the basics of building and deploying a bot using the Bot Builder SDK for .NET in some of the previous posts on this blog, and in the Bot Framework documentation. Once you have deployed a bot and registered it with the Bot Framework, you can enable your bot for the Direct Line channel: Ensure you capture your Bot Application ID during registration, and your Direct Line secret while enabling your bot on the channel. Next, we will create a basic full-window HTML page that makes use of the WebChat, based on the fullwindow example on GitHub: [snippet slug=agentbot1_bothtml line_numbers=false lang=html] After deploying this page to Azure Blob Storage, or an alternate web server, you should be able to access your bot like so: https://.blob.core.windows.net/mycontainer/usdagentbot.html?s=&botid=&username= (As a more secure alternative to using your Direct Line secret directly, you can use a Direct Line token, obtained by calling Direct Line's Generate Token.) In the Dynamics 365 Web Client, we now create a new Hosted Control for our bot, using the Standard Web Application type. You can opt to make your control Global, or Session-specific; in our example we will make it Global: We want to instantiate our control when USD is launched, so we create an Action Call to navigate our control to our bot URL: We then add this Action Call to the DesktopReady event of the Global Manager control. Now when we launch USD, we should see our bot in the RightPanel: Triggering Events and Actions in USD from our Bot Now that our bot appears in USD, let’s see how we can enable our bot to use the event model of USD to trigger actions in the desktop, by creating a custom event on our Bot Assistant control, then calling it via a URL that is opened by a Card Action in our bot. In this basic example, we will show how we can search for contacts by name, and allow our users to initiate a session with that contact. First, we will add a custom event to our Bot Assistant control, called ContactSessionRequested: To this event, we can call any actions we wish to in preparation for a new session, and include an Action Call called DoRoute from Bot. Note how we are including a Replacement Parameter that references a parameter from our event: This will trigger our Window Navigation Rule, which in turn will create our session and load our contact: We now need to call the event from our bot. USD provides a means of calling events on hosted controls using an event moniker syntax, as outlined in the Unified Service Desk documentation on user-defined events. We can use this syntax to call the event we created in USD: https://event/?eventname=ContactSessionRequested&contactid= We will now add code to our bot to allow the bot to call the URL above when appropriate. Making use of the code from our previous blog post that allows us to query Dynamics 365 data via the Web API, we can use the following code to use our bot dialog context, and a contact search term, to find matching contacts, and return them to our user. In this example, we are using a ThumbnailCard to present our information and CardAction objects to allow our user to select a button. When the button is selected, the URL will be opened, and our custom event will be called: [snippet slug=agentbot1_contact line_numbers=false lang=c-sharp] Included in the code is an overloaded ReadyForNextCommand method for reusability. Here is an example of how you can call this method in the context of a dialog that uses LUIS for natural language interaction: [snippet slug=agentbot1_luiscontact line_numbers=false lang=c-sharp] (see Enable language understanding with LUIS for more information on LUIS dialogs) When our Card Action button that is presented to the user is clicked, the event moniker URL will be called, and a session with the selected contact will be initiated: Stay tuned for more posts in this series, in which we will cover:

Making our bot aware of contextual information from USD
Enabling our bot to proactively notify our agent of alerts and insights based on the USD context
Enabling our bot for engagement via speech, in addition to typing
and more

Product and Part Identifier Bot using Custom Vision Service

Geoff Innis — Tue, 13 Jun 2017 18:11:04 GMT

In this post, we will build upon some of the capabilities that have been explored previously, including Microsoft’s Bot Framework, and the Cognitive Services Custom Vision Service. We will create a Bot that is capable of identifying a part or product from an organization’s Dynamics 365 product catalog, using an organization-specific image classifier that is powered by the Custom Vision Service. By enabling part or product identification through an intuitive and mobile-accessible bot, we can facilitate such scenarios as:

Enabling field technicians to quickly identify an unknown part or product while on-site, and providing them instant access to documentation and specifications
Enabling customers to quickly identify a part or product to facilitate service requests or product-specific self-service

This 40 second video shows the functionality in action: [video width="370" height="650" mp4="https://msdnshared.blob.core.windows.net/media/2017/06/CustomVisionBot3.mp4"][/video] We will build on the sample code available in these two resources:

The Receive Attachment code sample from the BotBuilder-Samples GitHub repository
The Custom Vision Service quickstart guide in the Cognitive Services documentation

Pre-requisites The pre-requisites for building and deploying our Product and Part Identifier Bot include:

An instance of Dynamics 365 for Customer Service (Online)
- You can request a trial of Dynamics 365 for Customer Service here
In our Dynamics 365 organization, we need to have configured:
- Products in our Products Catalog (ideally populated with product names, descriptions, product numbers, externally-accessible product image URLs, and links to product specifications)
Administrator credentials to access the O365 Admin Portal for our tenant that our Dynamics 365 instance is connected to
A Microsoft Azure subscription for application registration; A trial account will also work; See this previous post for a walkthrough of how to obtain Dynamics 365 Web API credentials
The Dynamics 365 Software Development Kit (SDK)
A Microsoft Account, for signing into the Custom Vision Service (https://customvision.ai)
Visual Studio 2015 or higher
Microsoft .NET Framework 4.6
The Bot Builder SDK for .NET
The Bot Framework Emulator (for Windows)
The Bot Connector Visual Studio Project Template
- Save the downloaded zip file to your Visual Studio 201X templates directory, which is typically found in %USERPROFILE%\Documents\Visual Studio 201X\Templates\ProjectTemplates\Visual C#

Setting Up our Custom Vision Service Project The first thing we will do is to set up our own Custom Vision Service project. The process of creating a Custom Vision Service project has been covered in a previous post, so to create our project, we can follow the steps outlined here (see Setting Up our Custom Vision Service Project). After we have created, trained and tested our classifier, we can obtain the necessary credentials from the Custom Vision site to call the Prediction API by selecting the Performance tab, and clicking Prediction URL. We can extract our Project GUID from the URL, and our Prediction Key from the Prediction-Key header value. Obtaining our Dynamics 365 Web API Credentials In order to query our Dynamics 365 Product Catalog, we will also need to have the required credentials to authenticate our requests via the Web API. In addition to an O365 username and password with sufficient D365 privileges to read our Product Catalog, we require:

Our Application Client ID
Our OAuth 2.0 Authorization Endpoint

See this previous post for a walkthrough of how to obtain these credentials (starting at Obtaining our CRM Web API Credentials). Building our Bot We will now start putting together the code for our bot, using steps found in the Create a bot with the Bot Builder SDK for .NET quickstart. We Open Visual Studio, and create a new project using the Bot Application template that we downloaded as a part of our pre-requisites: In our Project Properties, on the Application tab, we specify .NET Framework 4.6 as the target framework. Installing Packages and Adding References We need install several packages that we will leverage as a part of our bot code, and ensure our project is referencing the latest version of the Bot Builder SDK. We can do this via NuGet:

The latest version of the Microsoft.Bot.Builder package
Version 2.26 of the Microsoft.IdentityModel.Clients.ActiveDirectory package, to assist with Dynamics 365 Web API authentication

We also add references to the following assemblies, which we will use in our requests to the Custom Vision Service and Web API:

System.Net
System.Net.Http
System.Runtime.Serialization

We will place the Custom Vision Service credentials and Dynamics 365 Web API Credentials into our Web.config settings. We open the file, and add our credentials as appSettings as shown below, to make them accessible throughout our project: [snippet slug=customvisionbot_webconfig lang=xml] Adding a Data Contract for the Custom Vision Service To facilitate interacting with the Custom Vision Service, we add a new item to our project: a Visual C# class which we will name CustomVision.JSON.cs. We will populate this file with JSON data contracts for the data that comes back in the response from the sample code provided for the Prediction API. [snippet slug=customvision_datacontract lang=c-sharp] Querying Dynamics 365 via the Web API We will also create a separate class to hold our utility CRMWebAPIRequest method, used to issue our Web API requests to Dynamics 365, as done in previous posts:

The method accepts not only a request string, but also an HttpContent request payload, and request type indicator, to allow for not only querying, but creation of records in Dynamics 365
Dynamics 365 Web API credentials are retrieved from the Web.config
Depending on whether the request type, there is conditional logic to issue either a GET request when we are retrieving Dynamics data, or a POST request with content, if we are creating a Dynamics record

[snippet slug=customvisionbot_utilities lang=c-sharp] Next, we will adapt our MessagesController.cs class to be able to initiate our main dialog with the user. We replace the default Echo Bot code with code that will initiate a new Dialog using a VisionDialog class that we will create next: [snippet slug=customvisionbot_messagecontroller lang=c-sharp] Next, we add a new class to our project, which we call VisionDialog.cs. This class will manage the interactions with the Custom Vision Service and Dynamics 365 to identify our product. The code in the class will:

Present the user with the option to upload an image for identification using the StartingOptionsMessage method (Note: Options to allow dynamic credential setting and viewing are included as placeholder options)
If the user opts to upload an image, the AfterImageProvided method will
- Validate that an attachment has been included with the message
- Use HttpClient to retrieve the image as a byte array, using a JwtToken if necessary
- Issue a request to the Custom Vision Service, with the appropriate credentials
- Serialize the response using the CustomVision.JSON.cs data contract
- If a prediction is returned from the Custom Vision Service, a request will be made to the Dynamics 365 Web API to retrieve a matching product from the Catalog
- If a matching product is found in the catalog, a message with a Hero Card will be returned to the user, including product details, and a link to product specifications, if available

[snippet slug=customvisionbot_visiondialog lang=c-sharp] We are now ready to build and test our bot. As before, we press F5 in Visual Studio, which will Start Debugging. Once debugging has started, an Internet Explorer window should pop up, containing your bot endpoint information. We also launch our Bot Framework Emulator which we downloaded earlier. We take the URL from the IE pop-up address bar, append /api/messages, and enter this in the URL field of the emulator. We can use the MicrosoftAppId and MicrosoftAppPassword from the application template Web.config file, and add them in the appropriate inputs in the emulator as well. We can now test the identification of a product image through our bot emulator: Our bot is ready to be published to Microsoft Azure, registered with the Bot Framework, and made available across channels. If you wish to walk through the deploying of your bot from Visual Studio, see here.

Intelligently Routing Cases by Product with Cognitive Services Custom Vision

Geoff Innis — Tue, 23 May 2017 19:42:42 GMT

Microsoft’s Cognitive Services team recently made available a number of new services at the Build 2017 conference, to further augment the tools available to developers to enhance applications with natural methods of communication. One of these new services is the Custom Vision Service, currently in public preview, which “is an easy-to-use, customizable web service that learns to recognize specific content in imagery, powered by state-of-the-art machine learning neural networks that become smarter with training.” Domain-specific customizable image recognition has significant applicability to customer service scenarios, allowing us to intelligently automate the identification of products or parts, and to initiate product-specific actions. There are a number of ways we can apply this product or part recognition capability, including:

Auto-routing cases based on the product or part shown in an attached picture, in instances where the customer is not able to identify the product themselves (as we will show in this post)
Enabling field technicians to identify a part or product while in the field
Enabling customers to easily identify a product by simply taking a photo while engaging with a service-centric bot

In this post, we will see how the Custom Vision Service can be leveraged to automatically identify a product based on a photo that the customer uploads while submitting an issue via a self-service portal, then automatically and intelligently routed to the appropriate team with product expertise. We will create a Custom Workflow Activity that calls the Custom Vision Service to identify the product contained within an image attached to a Case Note. We will then use that activity within a Workflow process to automatically route the case to a product-specific queue. This 50 second video shows the functionality in action: [video width="800" height="450" mp4="https://msdnshared.blob.core.windows.net/media/2017/05/RouteCaseCustomVision.mp4"][/video] We will build on the sample code available in these two resources:

Create a custom workflow activity on MSDN, and the associated code samples in the Dynamics 365 SDK
The Custom Vision Service quickstart guide in the Cognitive Services documentation

Pre-requisites The pre-requisites for building and deploying our custom workflow activity include:

An instance of Dynamics 365 for Customer Service (Online)
- You can request a trial of Dynamics 365 for Customer Service here
In our Dynamics 365 organization, we need to have configured:
- Products in our Products Catalog
- Queues that align with our individual products
- A Dynamics 365 portal installed and enabled for case creation
- A case creation Entity Form that enables the attaching of a single file of the appropriate image MIME type / file extensions
The Dynamics 365 Software Development Kit (SDK)
A Microsoft Account, for signing into the Custom Vision Service (https://customvision.ai)
Visual Studio 2015 or higher
Microsoft .NET Framework 4.5.2

Setting Up our Custom Vision Service Project The first thing we will do is to set up our own Custom Vision Service project. The Custom Vision Service allows us to go beyond the broader image recognition capabilities of the Computer Vision API, and to create our own custom computer vision model, specific to the products or parts that we want it to be able to classify. The broad steps involved in preparing the Custom Vision Service are:

Signing into https://customvision.ai with your Microsoft Account
Building a Custom Vision Service project (Classifier)
Refining or Improving the Classifier
Testing the Classifier

The Custom Vision Service documentation provides an in-depth, step-by-step guide to this process, so rather than recreate it in this post, it is recommended that you follow the steps outlined here. We will hit on some of the key aspects as it relates to our product identification scenario. Following the instructions, we log into the Custom Vision Service, and create a new project. When creating a new project, we can choose a specific domain, or choose General. If your products are those typically found in a shopping catalog or shopping website, you may wish to select Retail. Otherwise, General may be more suitable: For each product or part that we wish to be able to identify, we will upload a variety of images, and assign a Tag to the images. While it is possible to assign multiple tags, we will assign a single tag to each image. We will use tags that match the names of the products as contained in the Dynamics 365 product catalog: After we have repeated the process of uploading and tagging photos for each of the products or parts that we will include in our classifier, we can proceed to train, evaluate, and improve our classifier as outlined in the quickstart guide. We also set our desired iteration as the ‘Default’ iteration. After training our classifier, we can also run a quick test with a local image file, or with an image URL: Assuming our results are satisfactory, we can proceed to creating a Custom Workflow Activity that will call the Prediction API to classify images attached to Notes. We can obtain the necessary credentials from the Custom Vision site to call the Prediction API by selecting the Performance tab, and clicking Prediction URL: We can extract our Project GUID from the URL, and our Prediction Key from the Prediction-Key header value. To use the Prediction API, we will build on the sample C# code provided in the quickstart guide. Building our Custom Workflow Activity In Visual Studio, we will first create a new project by selecting Workflow under Visual C# in the Installed Templates pane, and then select Activity Library. We will name our project DetermineProduct: In our Project Properties, on the Application tab, we specify .NET Framework 4.5.2 as the target framework. Adding References We add references to our project by right-clicking the DetermineProduct project in the Solution Explorer, and adding the following:

Microsoft.Xrm.Sdk
Microsoft.Xrm.Sdk.Workflow
System.Net
System.Net.Http
System.Runtime.Serialization

Note that the Microsoft.Xrm.Sdk and Microsoft.Xrm.Sdk.Workflow assemblies are found within the Dynamics 365 SDK. Adding a Data Contract for the Custom Vision Service To facilitate interacting with the Custom Vision Service, we add a new item to our project: a Visual C# class which we will name CustomVision.JSON.cs. We will populate this file with JSON data contracts for the data that comes back in the response from the sample code provided for the Prediction API. [snippet slug=customvision_datacontract lang=c-sharp] Adding Our C# Code Following the instructions outlined in the Dynamics 365 documentation, we delete the Activity1.xaml file in the project, and Add a new Class to the project, which we name DetermineProduct.cs. The full code for our class is shown further below, but first we will walk through the building of the class. To our new class, we:

add some using statements
make the class inherit from the CodeActivity class and give it a public access modifier
add functionality to the class by adding an Execute method

[snippet slug=customvision_usingstatements lang=c-sharp] We define our input and output parameters for our custom activity:

note_input as an input parameter
- this is the Note that has just been created to initiate our workflow
predictionKey as an input parameter
- this is the Prediction Key that we obtained earlier
projectGuid as an input parameter
- this is the identifier for our classifier project that we obtained earlier
product as an output parameter
- this is a Dynamics 365 EntityReference to the identified product from our catalog
probability as an output parameter
- this is an indicator of the confidence of the product identification

[snippet slug=customvision_parameters lang=c-sharp] In our Execute method, we create our Tracing Service and our Context for our custom workflow activity, then set our default return values: [snippet slug=customvision_tracingcontext lang=c-sharp] Next, we retrieve the note entity and required attributes based on the inbound note EntityReference: [snippet slug=customvision_getnote lang=c-sharp] We then confirm that we have an attachment filename, and if so, we convert the document body into a Byte array, for use with the Prediction API: [snippet slug=customvision_bytearray lang=c-sharp] Next, we create our HttpClient, prepare our request to the Prediction API with the appropriate headers, credentials, and URL structure, and retrieve our response: [snippet slug=customvision_createclient lang=c-sharp] We now:

serialize our response with our data contract
confirm that we have some Predictions of what the product is
use our top-ranking Prediction to build and execute a query to confirm that we have a matching product name in our catalog
set the matching product from the catalog as our output Product EntityReference, and output our prediction Probability as well

[snippet slug=customvision_callpredictions lang=c-sharp] The full code for our class is shown below: [snippet slug=customvision_fullsnippet lang=c-sharp] Before compiling our assembly, we sign it. In the project properties, under the Signing tab, we select Sign the assembly and provide a key file name. We are now ready to compile the assembly by Building the solution. Registering our Assembly We now need to register our custom workflow activity assembly on our Dynamics 365 instance. To do that, we will use the Plug-in Registration Tool. This tool is available in the Dynamics 365 SDK. Following the instructions in the documentation, we launch the tool, and authenticate using our administrator credentials for Dynamics. We then select Register New Assembly from the Register menu. In the resulting dialog box, we choose the location of our compiled assembly (which should be in the DetermineProduct\bin\Debug folder). We select our assembly and our workflow activity for registration. We specify Sandbox as the isolation mode, and Database as the storage location. Finally, we click Register Selected Plugins: We now have a custom workflow activity that will allow us to determine whether Note attachment images match a product in our catalog, and can use that as a part of our business processes. Routing Cases based on Product Identified in Image In our example, we will use our custom workflow activity to create a new workflow process that will attempt to find a product match for any cases created from the portal with an image attachment. Logged in to the Dynamics 365 web client with our administrator credentials, we navigate to Settings > Customizations > Customize the System. We choose Processes from the left navigation, and choose New. We specify that we are creating a Workflow type of process that is applicable to the Note entity, running in the background: When defining our process logic, we can now access our Determine Product action from the Add Step menu: We continue to design our process such that it is activated when a record is created, and we define our complete logic as described below:

we check to ensure our Note contains an attachment, is associated with a Case, and was submitted from the portal
if our check passes, we use our custom action to determine the product found in the attachment image
- note that we can restrict the MIME types that will be accepted in Case Creation attachments in our Entity Form settings of the portal
- we pass in our Note Entity Reference, and our Project and Prediction API credentials, as shown in the detail below
we check to see whether our first returned prediction has a probability of greater than the nominal value of 0.7; if so, we:
- set the appropriate product lookup value on our case
- add another note associated with our case to indicate that the product was auto-determined, and what the calculated probability was
- apply our routing rules; these routing rules can be configured to route to specific queues based on the product associated with the case

Detail on Custom Action Step Properties After we Save and Activate our process, we are now ready to test it. As shown in the video at the start of the post, we can create a case from our portal, attach a product image as we submit the case, and after a brief time, our case should be added to the appropriate product queue. With this functionality, we have been able to streamline the process of routing customer issues directly to subject matter experts in our organization, while enabling our customers to intelligently engage with us without requiring them to have extensive product knowledge.

Bringing Bots into an Omni-channel Customer Service Strategy – Part 2: Creating Cases

Geoff Innis — Thu, 28 Jul 2016 13:15:58 GMT

This post is the second in a series in which we explore how we can take advantage of the benefits of scale and channel breadth with bots, making sure the experiences we create with them build upon the investments that we have made in our customer service strategy, and feed into the consolidated customer 360 view using Microsoft Service and Dynamics CRM as our underlying customer engagement platform. We will build upon the bot we built in Part 1, which allows users to access self-service knowledge, tapping into the authoritative knowledge that has been curated and published using the robust knowledge management capabilities in the Dynamics CRM Interactive Service Hub. We will upgrade our bot from v1 of the Bot Framework to v3. We will also enable our bot to collect the necessary information to create a case in Dynamics CRM, for subsequent follow-up. We will use the concept of a FormFlow dialog within our bot to guide the user through a conversational experience to collect the information we require to submit our case to Dynamics CRM via the Web API. We will draw on the following resources:

Documentation on upgrading bots from v1 to v3 of the Microsoft Bot Framework
The AnnotatedSandwichBot sample bot application on GitHub

Note: It is recommended that you review the concepts in Part 1 of the series for an introduction and background. Pre-requisites The pre-requisites for building our service bot include:

An instance of Dynamics CRM Online; you can request a trial of Dynamics CRM Online here
Administrator credentials to access the O365 Admin Portal for our tenant that our CRM instance is connected to
A Microsoft Azure subscription for application registration; A trial account will also work; See Part 1 for a walkthrough of how to obtain CRM Web API credentials
Dynamics CRM credentials for a user with privileges to allow us to read Knowledge Articles and create Cases in CRM
Visual Studio 2015
- Update all Visual Studio extensions using Tools->Extensions and Updates->Updates
A Microsoft Account
The Bot Builder SDK
- You can download the SDK on GitHub here
The v3 Bot Framework Channel Emulator (for Windows)
- You can download the new Bot Framework Channel Emulator from here
The v3 Bot Connector Visual Studio Project Template
- You can download the template from here
- Save the downloaded zip file to your Visual Studio 2015 templates directory, which is typically found in %USERPROFILE%\Documents\Visual Studio 2015\Templates\ProjectTemplates\Visual C#

Updating our Bot to the Bot Framework v3 In this walkthrough, we will create a new project using the updated v3 Bot Connector Visual Studio Project Template, then bring in and adapt our v1 bot code. We Open Visual Studio, and create a new project using the new Bot Application template that we downloaded as a part of our pre-requisites. As before, we need install several packages that we will leverage as a part of our bot code. We can do this via NuGet. The packages include:

The Microsoft.Bot.Builder package
- Note that we no longer need to install the Microsoft.Bot.Connector package, as it has been consolidated into the Microsoft.Bot.Builder package
The latest version of the Microsoft.WindowsAzure.ConfigurationManager package
Version 2.26 of the Microsoft.IdentityModel.Clients.ActiveDirectory package, to assist with CRM Web API authentication
The Microsoft.Bot.Builder.FormFlow.Json package, to assist us with our FormFlow dialog

We will place the CRM Web API Credentials (obtained in Part 1 of the series) into our Web.config settings. We open the file, and add our credentials as appSettings as shown below, to make them accessible throughout our project: [snippet slug=botscasecreate_ lang=xml] Next, we will adapt our MessagesController.cs class to be able to initiate our main dialog with the user, just as in Part 1. With the v3 bot framework, however, the Message object has been replaced with the Activity object. We replace the default Echo Bot code with code that will initiate a new Dialog using the ServiceDialog class from Part 1: [snippet slug=botscasecreate_messagecontroller lang=c-sharp] This time around, we will separate our ServiceDialog into a separate class file. To our project, we Add a new C# class called ServiceDialog.cs. Into our class, we insert the code from Part 1, but we make some minor changes:

Any instances of Message objects are updated to Activity objects
We make use of the PostAsync method to send messages that are not prompting the user for information
We add two additional parameters to our call to the CRMWebAPIRequest method, to allow us to specify our HTTP Method, and pass a payload, to cater for our future requirements to create cases in our bot code
We set a value in our ConversationData with the Guid of one of our CRM Contacts
- As an extension of this example, we could add code to authenticate our user, and obtain their contact Guid, or alternately, create a new contact record based on data collected from the user

At this point, our updated ServiceDialog.cs class should appear as shown below: [snippet slug=botscasecreate_startservicedialog lang=c-sharp] We will also create a separate class to hold utility methods that may be accessed throughout our solution, such as the CRMWebAPIRequest method, used to issue our Web API requests to CRM. This method is taken from our Part 1 project, but adapted slightly, with these changes:

The method accepts not only a request string, but also an HttpContent request payload, and request type indicator, to allow for not only querying, but creation of records in CRM
CRM Web API credentials are retrieved from the Web.config app
Depending on whether the request type, there is conditional logic to issue either a GET request when we are retrieving CRM data, or a POST request with content, if we are creating a CRM record

[snippet slug=botscasecreate_utilities lang=c-sharp] With these changes, our bot will now function as before, enabling access to self-service knowledge using the v3 Bot Framework: Augmenting our Bot with Case Creation Capabilities We will now enhance the functionality of our bot by enabling end-users to log issues via a conversational bot experience, resulting in cases in Dynamics CRM. As noted earlier, we will leverage a different type of dialog called FormFlow. FormFlow dialogs offer the ability to gather the information required to achieve a particular objective in a simplified way, based on a definition you (the developer) create in either a C# class or JSON object. The Bot Framework documentation provides a thorough discussion of the use of FormFlow, and serves as a great reference for in-depth information. The gathering of information required to submit a case is a process well-suited to the FormFlow model. We will use the JSON Schema FormFlow in this walkthrough. One benefit of using this option is that, although in our basic example, we will be hard-coding certain attributes we will collect for our CRM Case within our JSON file, we could potentially adapt the entire set of data we need to capture at run-time, based on our CRM schema, through the dynamic creation of our JSON schema, allowing for extensibility. First, we will add a new file to our project, by right-clicking on the project in the Visual Studio Solution Explorer, then selecting Add… > New Item. We then select a JSON Schema File template, under the Visual C# > Web options: We also select our file in the Solution Explorer, and change the Build Action property to be Embedded Resource, to allow our file to be used to create our FormFlow dialog once built: In our JSON Schema File, we define the pieces of data that we need to collect. For each attribute we wish to collect, we are able to specify a prompt, a data type, options, validation, and other aspects. We define the information we wish to collect as described below:

a Title for the case: a string will be typed by the user
a Product that the case relates to: the user will be prompted to select from three options
- Note that the option values correspond to the Guid values of three of our products in our instance of CRM; we are able to present the appropriate Display Names for the products using the DescribeAttribute in our schema
- As an extension of this example, in a real-world scenario, these options could potentially be populated based on products the customer is known to own
a Priority for the case: the user will be prompted to select from four options
- Note that the option values correspond to the values of the Option Set used for the prioritycode field in our CRM instance; we again provide appropriate Display Names for the options
a Description of the issue being experienced: a string will be typed by the user

[snippet slug=botcasecreate_json lang=js] We now need to instantiate our Case Creation FormFlow dialog as a child dialog of our ServiceDialog at the appropriate juncture. To do this, we create a new method in our ServiceDialog.cs class, called BuildJsonForm, which will read the JSON schema we defined, and create and return a new JSON form builder: [snippet slug=botcasecreate_buildjsonform lang=c-sharp] Next, we will add a new CompletedCaseSubmission method to our ServiceDialog.cs class, which will process the JObject that is generated by the FormFlow dialog that gathers the case attributes. This method will receive the context and the case creation JObject as parameters, and will:

Retrieve the contacted Guid from conversation data; note that for our example, we are hardcoding a contactid in our Web.config appSettings, but in a real-world scenario, we would likely authenticate our user, or collect their contact information as a part of the dialog
Generate some JSON content for our request to the CRM Web API, to allow us to create an incident record in CRM, populating data we collected during our FormFlow dialog
- Important note: we include a custom caseorigincode in our example; If you wish to use the sample code with your instance of CRM, you will need to add a new caseorigincode Option Set Value of 100000001
Build a request Uri for our request to the CRM Web API (see CrmWebApiCSharp on GitHub for resources on interacting with the CRM Web API via C#)
Issue the request via our CRMWebAPIRequest method
Retrieve the resulting Guid for our new incident in CRM from the response
Use that new incident Guid to retrieve our newly-created customer-facing ticketnumber value via another “retrieve” request to the CRM Web API
Present the ticketnumber value back to our customer via a message back to the user
Finish the dialog using the IDialogStack.Done method

[snippet slug=botcasecreate_completedcasesubmission lang=c-sharp] We now adapt our AfterServiceChoiceAsync method, to create a child dialog to capture case details and create the case when the user selects Submit an Issue from the initial options presented. This is the new augmented method: [snippet slug=botcasecreate_afterservicechoiceasyncadapted lang=c-sharp] We are now ready to build and test our bot. As before, we press F5 in Visual Studio, which will Start Debugging. Once debugging has started, an Internet Explorer window should pop up, containing your bot endpoint information. We also launch our v3 Bot Framework Channel Emulator which we downloaded earlier. We take the URL from the IE pop-up address bar, append /api/messages, and enter this in the URL field of the emulator. We can use the default MicrosoftAppId (blank by default) and MicrosoftAppPassword (also blank by default) from the application template Web.config file, and add them in the appropriate inputs in the emulator as well. Note that these parameter names changed from v1 to v3. We can now have a conversation with our bot, via the emulator, and test out the submission of our case to CRM via the bot: The case has now been created and appears in CRM, and we can see that it originated from the bot: Note that if you wish to use a custom icon for cases originating from the bot, as shown above, see here for instructions. Our bot is again ready to be published to Microsoft Azure, registered with the Bot Framework, and made available across channels. If you wish to walk through the publishing of your bot and configuring of channels, you can find details here. Here is a brief video showing our bot’s behavior when published and deployed to the Skype channel: [video width="384" height="720" mp4="https://msdnshared.blob.core.windows.net/media/2016/07/ServiceBotV3_CaseSubmit.mp4"][/video] You can download the full source code for the solution here. Prior to building the sample, ensure you:

Populate your CRM Web API Credentials in the Web.config file
Populate a contactid Guid from your CRM instance in the MessageReceivedAsync method in ServiceDialog.cs
Populate three product Guids and product names in CreateCase.json
Add a new caseorigincode Option Set value of 100000001 via the Dynamics CRM Settings > Customization > Customize the System administration