Understanding Anomaly Detection

Let’s say you have a daily routine, you Wake up at 7:00am, Breakfast at 8:00am, Start work 9:00am, Lunch 1:00pm, Stop work 6:00pm., and Reach home 7:00pm You have been doing this for 5 years happily and your boss is also happy with your work. But, for a couple of days, you reach the office at 11:00am instead of 9:00am. Your boss and colleagues immediately ask you, “Is there anything wrong?”.  These two days are anomalies in your routine and human brain is trained to detect these anomalies because we don’t like them. Human brains are trained to normalize on patterns and let the routine ride your life.

Anomalies are everywhere, at work, at home, in industries, in your health, stock market, and also in computer systems. In cloud infrastructure, where every system is desired to have a consistent and predictable state, anomalies are like cancer. If you don’t detect them early on, they can spread and bring down your entire infrastructure. Usually, anomalies are associated with time-series (though not required) because we live in a time continuum and it is easier to map changes in values with time. Whether it is your heart beat or  memory usage of a server, mapping it to a time series will help you visualize and detect anomalies as they relate to time.

Below are some examples of anomalies detected using Twitter’s Anomaly Detection algorithm web service running in AzureML.

To understand how Twitter’s Anomaly algorithm works, please read this. If you want to run the R packages locally, I suggest you follow the instructions on Twitter’s blog. The procedure I followed was:

1. Read Twitter’s blog and studied how the algorithm works
2. Ran the R module locally in R studio
3. Packaged the R module using AzureML studio
4. Created an experiment
5. Published it as a web service
6. Created a C# client application to test the web service

Testing Twitter’s Code in AzureML

In AzureML, there is an Execute R Script module that lets you run a custom R script on an input data. For more information on how to get started with this module, please refer to this article.Next, I uploaded the sample data, zipped and uploaded Twitter’s R code and got the following experiment working in AzureML in no time.

I used the Project Columns module to project only the timestamp and count columns from the dataset to the R script because for anomaly detection, I only need timestamps and values for detecting anomalies.

Next, I modified the Execute R Script module script with the following code

# Map 1-based optional input ports to variables
raw_data <- maml.mapInputPort(1) # class: data.frame
# Contents of optional Zip port are in ./src/
source("src/date_utils.R");
source("src/detect_anoms.R");
source("src/plot_utils.R");
source("src/ts_anom_detection.R");

#Convert the first column to POSIXlt timestamp.

#Call Twitter’s R module function (from uploaded script)
res = AnomalyDetectionTs(raw_data, max_anoms=0.02, direction='both', plot=TRUE)

res$anoms[[1]] <- as.character(res$anoms[[1]], format="%Y-%m-%dT%I:%M:%S %Z")
resdf <- res$anoms
maml.mapOutputPort("resdf");

The script above acts as a wrapper over the Twitter’s AnomalyDetectionTs function. Notice how I am importing the dependent R files, modifying the time to POSIXlt format and then calling the AnomalyDetectionTs function from the ts_anom_detection.R file directly. Once you import the R files, functions from these files are available directly in the Execute R Script module. Once the anomalies are detected, I output the anomalies object (i.e. anoms) to the data output port (Port #1 – bottom left) . If you set plot=TRUE in the function, the plot get automatically output to the R Device port(Port #2 – bottom right). If you visualize both the ports, you will observe the following results (131 anomalies detected)

The blue dots represent the anomalies in the data. I could have been contented here and published the experiment as a web service. So, what was the problem?

I wouldn’t be able to expose all the parameters of the AnomalyDetectionTs() function as web service inputs. So, to do real justice to this fantastic algorithm, I decided to drive a bit further and create a custom R module in AzureML for this function.

Creating Custom R Module

Creating a custom R module in AzureML is not difficult as long as you understand the original R module that you want to wrap and how to create XML files. For more information on creating custom R modules, please visit this page.

For the AnomalyDetectionTs() function, I creates a new R file containing a wrapper function as shown below.

AnomalyDetectionTsw <- function(dataset1, max_anoms = 0.10, direction = 'pos',
                               alpha = 0.05, only_last = NULL, threshold = 'None',
                               e_value = FALSE, longterm = FALSE, piecewise_median_period_weeks = 2, plot = FALSE,  y_log = FALSE, xlabel = '', ylabel = 'count',  title = NULL, verbose=FALSE, narm = FALSE){

#Added by Tejaswi. Wrapper to work with AzureML
# Contents of optional Zip port are in ./src/
source("src/date_utils.R");
source("src/detect_anoms.R");
source("src/plot_utils.R");
source("src/vec_anom_detection.R");
source("src/ts_anom_detection.R");

if(only_last == "None"){ only_last <- NULL}
if(xlabel == "None") { xlabel <- ''}
 

dataset1[[1]] <- as.POSIXlt(dataset1[[1]])

 res <- AnomalyDetectionTs(dataset1, max_anoms, direction,
                               alpha, only_last, threshold,
                               e_value, longterm, piecewise_median_period_weeks, plot,
                               y_log, xlabel, ylabel,
                               title, verbose, narm)
 
res$anoms[[1]] <- as.character(res$anoms[[1]], format="%Y-%m-%dT%I:%M:%S %Z")

 if(plot == TRUE){

print(res$plot)

}
 
 return(res$anoms)
 
}

Observe the similarity between the Execute R script and the new R module above. With this R module, I can now load the dependent modules, call the AnomalyDetectionTs() function, and most importantly expose the function parameters as web service parameters. The definition of input parameters and entry point of the module is defined in an XML file. Next, create the XML file for the wrapper module as shown here.

You can find the final wrapper source code and the XML file I created here.

Now zip-up the original content (all Twitter modules), the wrapper module, and the xml file into one zip file and upload it as a module in AzureML Studio.

After processing and validating, the custom module should show up in the Custom section of the Toolbox of the AzureML Studio

Testing the R Module

Now you have your own Anomaly Detection module you can use in any experiment. Isn’t that cool? You can use and test the module like any other module in AzureML Studio.

If you test the module with the original dataset, it should yield the exact same results as before (131 anomalies). Note that we are not changing any algorithm logic, merely wrapping it into a reusable module.

Publishing Web Service

Now, you can create a web service and publish it that any client app can call. A couple of things to keep in mind:

1) Select the web service parameters you want to expose from the properties pane of the custom module. These parameters are defined in the XML file of the module.

2) Provide default values for the appropriate parameters

3) Make sure you have one input and two Web Service outputs (not required) so that you can send even the plot all the way to the end user.

The Client Application

Finally, we will need a client application to consume the web service. I have already built one for you, so relax. Please download the source code for the C# client console application from here. Before running the application, please update the AnomalyClient.exe.config file with the Url and Access Key of your AzureML web service.

<add key="AnomalyDetectionWebServiceUrl" value="" />
<add key="AnomalyDetectionApiKey" value="" />

To display the application parameters/switches, type the following on the command prompt.

>AnomalyClient.exe --help

You may also run the runtests.cmd file from the command prompt to execute a few tests. The anomalies are created as [GUID].csv files and plots are created as [GUID]-n.png; where [GUID] is a random GUID and n is the count of graphic (currently only 1).  The logs will be available in the output.log file.

Note: The application lets you input a CSV file as an input with two columns “timestamp”, “values”.  Timestamp must be the first column and Values must be the second column. If you have it reversed, it won’t work. If you don’t specify any input file, the application will generate random data.

The application should work with or without headers in the CSV file. Use it at your own risk, but enjoy it thoroughly!

Thank you!

Tejaswi

Source Code Repo

References

Twitter’s Anomaly Detection Algorithm

Twitter’s Anomaly Detection GitHub Repository

Anomaly Detection with Twitter in R

Twitter’s Anomaly Detection Package

Seasonal Hybrid ESD

With tons of new features coming to Windows 10 and 1000s of new API capabilities, I though it would be great to put all the training together in one blog that not only me, but anyone can refer to. The links inlucde basic enhancements like XAML bindings to light-up features like Cortana integration. Enjoy!

Microsoft Virtual Academy Course

A Developer's Guide to Windows 10

XAML

What's New in XAML for Universal Windows Apps

Developing Universal Windows Apps in Visual Studio 2015

Harnessing the Full Power of Input in Universal Windows Apps

Universal Commanding and Navigation in Your XAML

Data Binding: Boost Your Apps' Performance Through New Enhancements to XAML Data Binding

Deep Dive into XAML and .NET Universal Windows App Development

XAML Performance: Techniques for Maximizing Universal Windows App Experiences Built with XAML

Moving to the Universal Windows Platform: Porting an App from Windows 8.1 XAML or Windows Phone Silverlight to Windows 10

New XAML Tools in Visual Studio 2015

XAML Case Study: Putting it All Together, Office and XAML

JavaScript

JavaScript Frameworks in Your Apps and Sites from WinJS and Beyond

UX, Layout and Scaling

Designing and Developing the Ultimate Windows App Experience

Display Scaling: What It Is and What You Need to Know About it to Have Great Visuals

Windows Design for Developers: Getting the Balance Right

Design: UX Patterns and Responsive Techniques for Universal Windows Apps

Surface Hub: Building Windows Universal Apps for Surface Hub and the Large Screen

Bring Fluid, Responsive, and Highly Scalable UI Experiences to Your Universal Windows Apps with the New Visual Layer

From the Small Screen to the Big Screen: Building Universal Windows App Experiences with XAML

Optimizing Windows Apps for Continuum

Screen Casting: Develop Multi-Screen Universal Windows Apps Using Casting Technologies

Use New Motion and Effects Features to Captivate Users and Truly Bring Your App to Life

Tiles, Notifications, and Action Center

App Lifecycle

App Lifecycle: From Activation and Suspension to Background Execution and Multitasking in Universal Windows Apps

Introducing the Windows 10 App Model

App Analytics

Application Insights for Any App: A Must-Have Tool for Understanding Your Customers

Fast and Powerful Diagnostics, and Problem Solving with Application Insights

Cortana and Speech

Cortana Extensibility in Universal Windows Apps

Cortana and Speech Platform In Depth

Inking

Harness the Full Power of Digital Inking in Your Universal Windows App with Ink Recognition, Advanced Ink Processing, and More

Introducing DirectInk: Learn How to Unlock New Opportunities Using Ink in Your App

Face, Speech and Vision APIs (Project Oxford)

"Project Oxford": Adding Smart to Your Applications

Vision APIs: Understanding Images in Your App

Maps

Leveraging Maps and Location Across Phone, Tablet, and PC in Your Windows Apps

Build with Bing: Use Bing APIs and Tools to Enrich Your Apps and Make Them More Discoverable

Audio, Camera & Video

A Studio in the Palm of Your Hand: Developing Audio and Video Creation Apps for Windows 10

Camera: Developing Powerful Camera Apps

Developing Audio and Video Apps

User Data and Authentication

User Data: Working with Contacts, Appointments, Text Messages and More on Windows 10

Microsoft Passport and Windows Hello: Moving Beyond Passwords and Credential Theft

Windows Store

All That is New in the Windows Store

Store: Deep Dive on Publishing Universal Windows Apps

App Packaging and Deployment for Universal Windows Apps

Acquiring Users and Monetizing Your Apps

API Contracts (or How I Learned to Stop Checking OS Versions and Love Feature Detection)

Monetize and Promote Your Universal Windows App with Ads

Hosted Web Apps

Hosted Web Apps and Web Platform Innovations

Localization

Cross-Platform Localization with the Multilingual App Toolkit

Sensors and Printing

Building Rich, Contextually Aware Universal Windows Apps Using Sensors

Printing: Developing Apps That Print in Windows 10

Building Compelling Bluetooth Apps in Windows 10

App Services

App-to-App Communication: Building a Web of Apps

Text Analytics is the process of transforming text into information and actionable output. Text is prevalent in all the industries. If something doesn’t already exist in text format, it ultimately lands up as text for consumable purposes. Most of the Machine Learning algorithms depend on CSV and JSON formats, which are object representations in textual format. What we speak also gets converted to text when it comes to speech recognition applications. Why? Because it is easier for applications to parse and consume text. Understanding the context is a different story, and that is where Natural Language Processing comes into play by transforming text into understandable corpora and lexicons. What does Microsoft have to offer in Text Analytics? Combined, our applications process more text than anyone else in the world (Office and Bing).

The AzureML Team has produced a series of experiments for building Text Classification models.

Watch this video: Learn How to Create Text Analytics Solutions with Azure Machine Learning Templates

Text Classification: Step 1 Data preparation

Loading, editing, cleaning, partitioning, and filtering the dataset is covered in this module.

AzureML Gallery Link

Step 2: Text preprocessing

In the preprocessing step, this experiment demonstrates the importance of processing the text for cleaning the dataset. Some of the preprocessing examples include removing special characters, assigning contextual meaning to special characters and text symbols (e.g. J, LOL), removing duplicates, punctuations, and stop-words.

AzureML Gallery Link

Step 3: Feature engineering

Step 3 has 2 parts:

3A: n-grams TF-feature Extraction

3B: unigrams TF-IDF feature extraction

Now that you have the data cleaned up, it’s time to extract features.

For a mathematical machine learning algorithm, textual features doesn’t make much sense, therefore, this module demonstrates the use of Feature Hashing to convert variable length text into numeric feature vectors. When its numbers, Math is happy. The step also demonstrates how to simplify the dimensions of the feature vectors using the “Filter Based Feature Selection” module.

Step 4: Train and evaluate models

The hard part is over. In this module, you select your favorite algorithm(s), and train the machine learning model.

AzureML Gallery Link

Step 5: Deploy trained models as web services

Step 5 has two parts:

5A: Deploy TF Web Service

5B: Deploy TF-IDF Web Service

And finally, you deploy the web services to be used in your applications.

Text Analytics Web Service

If you don’t want to build an ML service from scratch, the AzureML team has also published a TextAnalytics web service in the Azure Datamarket with sample code and documentation.

https://datamarket.azure.com/dataset/amla/text-analytics

Documentation

Sample Code

Text Analytics and Vowpal Wabbit in Azure Machine Learning Studio

https://azure.microsoft.com/en-us/documentation/videos/text-analytics-and-vowpal-wabbit-in-azure-ml-studio/

Wouldn’t it be nice to have all your Machine Learning (ML) tools available wherever you are, and on any device?  Azure RemoteApp is built exactly for that. While traveling, I usually carry my Surface Pro 3 that does not have any ML tools,  and at home I have my monster ThinkPad W350 with all the ML tools,  as well as an iMac to work on all the cool Open Source and Mac-friendly Microsoft software. I would like to use my ML tools on-the-go from airplanes to boring meetings. In this article, I will describe the process of creating a custom Azure RemoteApp template with your favorite Machine Learning tools installed, and then streaming these tools to any device. Ready for the ride?

Pre-Requisites

1. Azure Subscription with Access to RemoteApp
2. Knowledge of creating Azure Virtual Machines

Which Virtual Machine to use?

The RemoteApp team has created a “Windows Remote Desktop Session Host” server image for your convenience to quickly get started with the right configuration required by a RemoteApp custom template. If you want to start from scratch, here are more details.

Login to Azure portal and create a new “Windows Remote Desktop Session Host” VM

Below is the VM configuration I used for this exercise.

Installing Machine Learning Software

Once the VM is ready, login to the VM and install your favorite Machine Learning client tools. I installed the following ones:

• Python
• Anaconda
• R
• R Studio
• Notepad++, SublimeText (for text editing if required).

You can install more of your favorite tools if you like.

Start Menu Requirement

After installation is complete, add the installed software to the Start Menu by right-clicking and selecting “Pin to Start Menu”.  RemoteApp will pick up the software listed in the Start Menu.

My Start Menu looks like this.

Validating Image

Next, the RemoteApp team has created a nice tool called “ValidateRemoteAppImage” for validating the VM for RemoteApp publishing. This tool is available on the Desktop of the “Windows Remote Desktop Session Host” VM image.

Double-click the tool to validate the VM (as shown below). If there are errors, fix them as suggested.

Capturing the Image

RemoteApp requires captured VM images to publish, deploy, and stream the applications. Follow the procedure listed here to capture the VM image. Make sure you generalize and shutdown during sysprep as shown below.

After the image is shutdown, click the capture button from the Azure drawer menu for the VM.

After the image is captured, the VM will be deleted and the image will appear in the Images section of the portal

Adding Image to the RemoteApp Template library

Next, we need to let the RemoteApp know that we want to use this VM image for creating our collection. Go to the Template Images section of RemoteApps and click on “Upload or Import a Template Image”.

Next, click on “Import an image from your Virtual Machine library”

On the “Import a Template Image” page, select the VM image you created earlier.

On the next screen, give your VM Template image a name and select the location. After successful import, the template will be listed in the “Template Images” section of RemoteApp.

Creating a Remote App Cloud Collection

The custom VM template portion of the exercise is now complete. Create a new RemoteApp collection as shown below

In the Template Image dropdown, you will see your VM image listed. Please specify your preferred name, region, and plan (Standard Recommended for Machine Learning Tools) for the collection. The collection should get listed in the “RemoteApp Collections” section of the portal.

Publishing Applications

Remember those apps that you added to the start menu earlier? We need to publish those to RemoteApp individually. On the RemoteApp Collection page, click “Publish remoteapp programs” and select the apps you want to publish.

Adding Users

Next, to add users, click on “Configure user access” and add the identities of the users. For more information on supported user identities, please refer to the following link.

https://azure.microsoft.com/en-us/documentation/articles/remoteapp-user/

Running the Remote App

Our Remote App is now ready, lets start using it. Download the RemoteApp client from https://www.remoteapp.windowsazure.com

For Mac users, the website will automatically redirect you to the Mac App Store for downloading “Microsoft Remote Desktop” app. The Microsoft Remote Desktop App has a menu item for Azure RemoteApp.

Install and login to the Azure Remote App client using one of the user access accounts you created earlier. On successful login, you should see the apps you published in the Azure RemoteApp console.

Running the Apps

Double-click on any of these tools to run on any supported device. Let’s start with Anaconda, a scientific Python distribution.

NLTK Dispersion Plot

Start Anaconda command prompt and type python to enter Python’s interactive environment. NLTK is packaged with Anaconda distribution, but you still need to download the required corpora for testing. As shown below, download all the NLTK corpora using NLTK downloader.

Note the Download directory. Every user has a specific directory on the RemoteApp VM. There is nothing magic going on behind the scenes. The RemoteApp infrastructure runs an instance of the VM template we created and then streams the published apps to the end-user. Every permitted user gets an account and shares the same VM instance.

Once the NLTK downloading is complete, we can start testing it.

Import the NLTK book corpora from the command prompt as shown below.

>>> from nltk.book import *
*** Introductory Examples for the NLTK Book ***
Loading text1, ..., text9 and sent1, ..., sent9
Type the name of the text or sentence to view it.
Type: 'texts()' or 'sents()' to list the materials.
text1: Moby Dick by Herman Melville 1851
text2: Sense and Sensibility by Jane Austen 1811
text3: The Book of Genesis
text4: Inaugural Address Corpus
text5: Chat Corpus
text6: Monty Python and the Holy Grail
text7: Wall Street Journal
text8: Personals Corpus
text9: The Man Who Was Thursday by G . K . Chesterton 1908
>>>

Next, let’s create a dispersion plot of certain words against specific corpora. text4 is the US Presidential inaugural address corpora, so lets try to find out common words spoken by all the presidents.

>>> text4.dispersion_plot(["citizens", "democracy", "freedom", "duties", "America", "dream", "independence", "government", "democrat", "republican"])

From the figure above, you can derive interesting insights from the textual content. If you run the same words against another corpora, what would the dispersion plot look like? 

For more information on NLTK, please visit https://www.nltk.org

R Text Analytics

Now let’s try R, another popular language with Data Scientists. In this exercise, lets try to build a pie chart of the types of client applications used for tweeting “#microsoft”. I have used the sample from the twitteR package documentation (Source: https://geoffjentry.hexdump.org/twitteR.pdf  )

Start R Studio and install twitteR package from Tools > Install Packages

Setting up Twitter Client

Use this link for setting up and accessing Twitter API using the twitteR client. Once the client is setup and tested, we will call the APIs as listed below for building our pie chart. In R Studio, type the following.

> r_tweets<-searchTwitter("#microsoft",n=3000)
> sources<-sapply(r_tweets,function(x)x$getStatusSource())
> head(sources)
[1] "<a href=\"https://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>"
[2] "<a href=\"https://ifttt.com\" rel=\"nofollow\">IFTTT</a>"              
[3] "<a href=\"https://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>"
[4] "<a href=\"https://dlvr.it\" rel=\"nofollow\">dlvr.it</a>"              
[5] "<a href=\"https://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>"
[6] "<a href=\"https://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>"
> sources<-gsub("</a>","",sources)
> sources<-strsplit(sources,">")
> sources<-sapply(sources,function(x)ifelse(length(x)>1,x[2],x[1]))
> head(sources)
[1] "Twitter Web Client" "IFTTT"              "Twitter Web Client" "dlvr.it"          
[5] "Twitter Web Client" "Twitter Web Client"
> source_table=table(sources)
> pie(source_table[source_table>50])

Conclusion

RemoteApp is one of the killer apps for Azure. It will save you significant amount of time and money just by moving your desktop apps to a service consumption model. It's not just the cost that’s driving the adoption, but also its scale and reach. RemoteApp will be available in all the Azure datacenters around the world. Imagine building this service yourself from scratch with similar scale and global reach. The scenario mentioned in this article is just scratching the surface of enhancing worker productivity through Azure RemoteApp.

References and Resources

Text Analytics using R

https://www.rdatamining.com/examples/text-mining 

twitteR documentation

https://geoffjentry.hexdump.org/twitteR.pdf 

RemoteApp Resources

Creating a RemoteApp Template Image

https://blogs.msdn.com/b/rds/archive/2015/03/17/how-to-create-a-remoteapp-template-image-in-azure.aspx

Creating Cloud Collection

https://azure.microsoft.com/en-us/documentation/articles/remoteapp-create-cloud-deployment/ 

Creating Custom Image

https://azure.microsoft.com/en-us/documentation/articles/remoteapp-create-custom-image/

Creating Hybrid Collection

https://azure.microsoft.com/en-us/documentation/articles/remoteapp-create-hybrid-deployment/ 

Azure Remote App Forum

Azure RemoteApp forum.

Azure RemoteApp Documentation

https://azure.microsoft.com/en-us/documentation/services/remoteapp/

Recently I have been spending some time on Speech Recognition. The Speech Recognition Grammar Specification (SRGS) file is typically used for defining speech commands that a user might utter. It works fine when the commands are known and you just want to capture it and in turn execute it. But there are scenarios where you want to capture everything that the user is speaking and then retrieve entities (e.g. nouns [ORG, PERSON, PLACE, etc.]) from the text. For example, in the financial scenario I am working on, I want to capture company names from user’s speech. Running this kind of algorithm locally can be resource consuming on the device (especially mobile). Introducing AzureML’s Named Entity Recognition.

Named Entity Recognition (NER) is a Text Analytics module in AzureML that does exactly what I want here. It will extract entities from text and tag them as people (PER), location (LOC), or organization (ORG). NER is probably the simplest module available in AzureML.

Step 1: Create Data

I just mocked up some training data in a text file and named it namedentitytrain.tsv. The contents of the file are as follows

search Microsoft.
show Microsoft.
get Microsoft.
show Microsoft details.

I am trying to train the NER module with similar text that I am expecting a user to speak. Note that this is a very simple dataset for blog purposes only. In reality, I will be adding more combinations of text utterances to train the module.

Step 2: Store Data

Next, I stored the data in Azure Blob Storage (you can store is anywhere) and made it accessible via URL.

Step 3: Start the experiment

Once I had my data in place, I opened the AzureML studio https://studio.azureml.net and created a new experiment

Step 4: Specifying the Data Input

In order to make your training data available to the experiment, you must use either an existing saved dataset or a Reader module

Next, I configured the Reader module to point to the TSV data file I had created earlier

I follow the best practice of visualizing the data at every stage of the process.

Good, now the experiment can view the data. Note, you may have to run the experiment before visualizing it.

Step 5: Configure NER Module

From the Text Analytics section, I simply dragged the NER module and then connected the Reader to its left input. Next, I ran the experiment and connected the Web Service Input and Output modules to the NER input and output.

I ran the experiment again and published the web service by clicking Publish Web Service button

Step 6: Testing the Web Service

Once the Web Service is published, you are free to test it.

The Text link opens up a popup for you to enter web service parameters. I entered “search oracle”. Now, I expect the model to be trained and return be a tagged data.

Result:

Voila!

I can now retrieve named entities from any text. My next step is to tie this to Speech Recognition and then I should be able to retrieve entities spoken by a user during a conversation.

For more information on the NER return values, please refer to

https://msdn.microsoft.com/en-US/library/azure/dn905955

Isn’t Machine Learning Fun?

Thanks,

Tejaswi Redkar

After speaking with several developers and partners around the world, I decided to blog this instead of sending emails to all of them every time. Please let me know if I am missing a critical resource.

Since Office has adopted the new HTML + JS model for developing Office Apps, there has been a lot of demand in learning this model and understanding how it fits in the traditional model. The Office Store App model is designed for portability of your apps across multiple kinds of OS. The same app will run wherever Office runs. My personal favorite is the Outlook Contextual App because it enables new scenarios and ways of reaching to your end-user.

When it comes to Windows, HTML + JS is a boon. I have seen a D3.js visualization working in a native Windows Store App like a charm in no time.

Enjoy!

Microsoft Virtual Academy

Developing Universal Windows Apps with HTML and JavaScript Jump Start

https://www.microsoftvirtualacademy.com/training-courses/developing-universal-windows-apps-with-html-and-javascript-jump-start

Windows 8.1 UX Design Jump Start

https://www.microsoftvirtualacademy.com/training-courses/windows-8-1-ux-design-jump-start

Windows Store Apps with HTML5 Refresh Jump Start

https://www.microsoftvirtualacademy.com/training-courses/windows-store-apps-with-html5-refresh-jump-start

Advanced Windows Store App Development with HTML5 Jump Start

https://www.microsoftvirtualacademy.com/training-courses/advanced-windows-store-app-development-with-html5

Cross-Platform Development with Visual Studio

https://www.microsoftvirtualacademy.com/training-courses/cross-platform-development-with-visual-studio

Samples

Universal App Samples (In C++, C#, and JavaScript)

https://code.msdn.microsoft.com/Universal-Windows-app-cb3248c3

Free Book (a little dated but the concepts are still valuable)

https://blogs.msdn.com/b/microsoft_press/archive/2012/08/20/free-ebook-programming-windows-8-apps-with-html-css-and-javascript-second-preview.aspx

Office 365 Apps

Deep Dive: Integrate Office 365 APIs in Your Mobile Device Apps

https://www.microsoftvirtualacademy.com/training-courses/deep-dive-integrate-office-365-apis-in-your-mobile-device-apps

Introduction to Office 365 Development

https://www.microsoftvirtualacademy.com/training-courses/introduction-to-office-365-development

Finally, there are two sites that cover these topics in detail including design and development guidance.

https://dev.windows.com

https://dev.office.com

Have fun developing!

Tejaswi

I was running some passive speech recognition code in a Windows Phone project and I kept getting the following error

The speech privacy policy was not accepted prior to attempting a speech recognition

I was using the non-UI recognizer for passive listening on a thread.

SpeechRecognizer spchRecog = new SpeechRecognizer();

SpeechRecognitionResult spchResult = await spchRecog .RecognizeAsync();

instead of

SpeechRecognitionResult spchResult = await spchRecog .RecognizeWithUIAsync();

Found the privacy policy error on MSDN, and implemented the recommend exception handling.

https://msdn.microsoft.com/library/windows/apps/jj662934(v=vs.105).aspx#BKMK_Handlingthespeechprivacypolicyerror

In the exception handling code on MSDN, the default error message shown to the user is

“You will need to accept the speech privacy policy in order to use speech recognition in this app.”

Now imagine my mom using this application and she gets this message. There is no reference to what does it mean by speech privacy policy. My mom will get confused, and she does use a Windows Phone.

Actually, you have to just let the user know to enable the speech recognition service by navigating to

Home->Settings->Speech->Enable Speech Recognition Service

Ideally, the error message should automatically navigate the user to the appropriate setting. But that’s a long shot. In the short term, I modified the error message to

“In order to use speech recognition, you will need to enable the speech recognition service by navigating to Home Page->Settings->Speech->Enable Speech Recognition Service.”

Good enough for my mom for now. If you see any cryptic error message regarding speech privacy policy, first check whether the Speech Recognition Service is enabled on the phone or not.

That’s it! The recognizer to work like a charm.

Thanks,

Tejaswi

What is AzureML?

AzureML is meant to democratize Machine Learning and build a new ecosystem and marketplace for monetizing algorithms.  You can find more information about AzureML here.

Why AzureML?

Because it is one of the simplest tools to use for Machine Learning. AzureML reduces the barriers to entry for anyone who wants to try out Machine Learning. You don’t have to be a data scientist to build Machine Learning models anymore.

Logical Machine Learning Flow

Figure below illustrates a typical machine learning process with end result in mind.

1) Define Objective: Machine Learning is useless without a clear objective for the experiment. Once the objective is clear, then you need to search for the data. In this exercise, our objective is to predict the price of a car based on some attributes provided by an end-user.

2) Collect Data: During this phase, you vehemently collect and ingest the data into a data repository. For this exercise, we will use a sample automobile dataset from the University of California, Irvine’s Machine Learning Repository.

https://archive.ics.uci.edu/ml/datasets/Automobile

Note: This dataset is old and the values no longer represent the values of today’s cars. In a future article, I will show how to feed a real-world automobile feed into the model.

3) Prepare Data: In this phase, you prepare the data for modeling by identifying the features in the dataset. Flatten the data to represent one row per record, remove any outliers, and split the data for training and testing your models. Sometimes, you may have to normalize the data.

4) Develop Model: Designing a model is the fun part, especially using popular ML modeling tool like AzureML, Weka, Lionoso, Orange, MatLab, Octave, R studio, etc. When you develop a model, you feed the data to one or more ML algorithms you have selected for your experiment. In our experiment, we have “true/valid” data and the output will be the predicted price of the car i.e. real-number (or continuous value).

5) Train Model: Typically, supervised learning models need to be trained with data that has already been validated. For example, the car data that we have is “true” data and the output will be a predicted singular value. During training, the ML model operates on the training dataset and builds a pattern that will be used for processing the test and live datasets. You must always keep training and testing datasets separate with minimal overlap.

6) Analyze/Evaluate: Once the model is trained, you can then score the model to generate evaluation metrics and pass it for evaluation. During evaluation, you validate the performance of your trained model and decide whether you want to proceed with it or make adjustments. This is where the experiment gets interesting. If you are not happy with the evaluation, you can readjust the features, change your algorithm for comparison, or even start a brand new experiment.

7) Monitor/Operate: This is where AzureML adds value and empowers not only a data scientist, but also a developer to quickly Publish, Monitor, and even Monetize their algorithms.

Now, lets follow the process described above to build our experiment.

Create Workspace

Navigate your browser to https://www.azure.com and under Feature section, select Analytics –> Machine Learning or go directly here.

Click on Get Started.

There are three steps to start working on AzureML.

You need an Azure account. Microsoft offers free 3 month trials and it is the best way to get started on any Azure service. Once you have an account, login to the Azure Portal.

Create a new Machine Learning workspace.  New –>Data Services –> Machine Learning

A workspace is used as a collection of experiments that you will be working on. Your workspace should show up in the list of machine learning workspaces. Click on your workspace to go to the ML dashboard.

Sign-In to ML Studio

On the dashboard, client on Sign-in to ML Studio. The ML Studio is where you will create your experiments and torture the data.

We have not setup our ML tool for creating experiments. This is just one time setup and now you are free to create as many experiments as you wish. Referring back to our ML flow, we only defined the objective, now we need to collect the data.

Upload Dataset

First download imports-85.data the automobile dataset from https://archive.ics.uci.edu/ml/datasets/Automobile

This data is old, but the purpose is to learn the tool. In a future blog I will show how to get a stream of automobile data and then predict prices based on that.

Click on New-> Dataset –> From Local File

Note: In AzureML, you can also read a dataset from a URL directly into your experiment. By uploading the dataset you are saving it for future use.

Create a new Experiment

Click on New –> Experiment –> Blank Experiment

This will create a new experiment canvas with a tool box on the left for you to drag and drop components

Connect Data

Your Automobile dataset should appear in the Saved Datasets section of the toolbox. Drag and drop the automobile data onto the canvas.

Note that the data component has a connection point to connect to other components. If you have used any Workflow or ETL tools before, this should sound familiar to you.

Prepare Data

I usually follow the best practice to test my experiment at every stage and on every change I make. So, right-click on the connection point and client Visualize to see the list of raw automobile data.

But, there is a problem. Some of the data is missing. This might create adverse effects in predicting the car value. And we also don’t need the normalized-losses column (feature) for prediction purposes. That column is used for insurance information. So, to get rid of that column, drag Project Columns component from Data Transformation –> Manipulation.

Project Columns component lets you filter the columns from one stage to another. In our case, we will not project the normalized-losses column at all.

Next, click the Launch column selector from the Properties (right-side ) pane

In the Select Columns window, select “All columns” to select all columns and then select “Exclude” –> “column names”, and add “normalized-losses” as shown in the figure below.

To validate, save the project, and Visualize the Project Column’s output connection point.

If you observe carefully, there are some missing values in some columns and these values may create unnecessary noise in our prediction, therefore, let’s remove all the rows containing missing values.

To remove missing values, drag the missing values scrubber and connect Project Columns to Missing Values Scrubber. Select “Remove entire row” from the “For missing values” dropdown list.

Next, Save, Run and Visualize to validate whether your dataset consists of all the values. Note the number of rows and columns displayed to compare them with the original dataset.

Now, we have scrubbed the dataset, but considering our objective of predicting the car price, there are still many columns that may not be a contributing factor to the price of a car. A simple question I would is: “What affects the price of a car?”. Now, remember that this is an experiment and your answer will be different than mine. My hypothesis is that the following columns from the dataset affect the price of a car.

make, body-style, wheel-base, engine-size, horsepower, peak-rpm, highway-mpg, num-of-cylinders

Add another Project Columns component, Launch Column selector and select the listed columns to project.

Run and Visualize the data. You should see only the selected columns displayed.

Split Data

The data is now prepared to be fed to the model. But, in ML, you must first train the model and then test the hypothesis with test data that is different from the training data. AzureML provides you with a Split function to split the data into two paths. Add the Split component and connect it to the Project Columns as shown in the Figure below.

It is a common practice to split the data 75% (training) and 25% (testing).

Train a Linear Regression Model

Next, drag the Linear Regression, Train Model, and Score Model components. The Score Model component generates predictions on the test data by using the trained model. Connect the Linear Regression and Split components to the Train Model. Select the price column as the outcome column. The outcome column consists the predicted value.

Connect the Train Model to the Score Model. Next, connect the second output of the Split component to the Score Model. The output of the Score Model is sent to the Evaluate Model component for generating Evaluation metrics. The finished model looks like the figure below.

Run the model and Visualize the Evaluate node. The Evaluation results will consist of a table with the following columns:

Coefficient of Determination (R2): Determines how well your regression line fits the statistical data. The value of R2 is between 0 and 1 with 1 being a perfect fit. In our case, R2 is approximately 82%. Whether this value is good or bad depends on the error tolerance for your output. In automobile prediction, I am happy with 82% success. If you want to improve the results further, you can modify the Project Columns and also the settings for the Liner Regression. You may also try another algorithm like Bayesian Linear Regression. You can either create a new experiment or add Bayesian Linear Regression component to the current experiment and feed it exactly the same data as before.

When you run the experiment, it runs both the models and evaluation is done separately. In our experiment, R2 came out to be 89%.

Now assuming that we are happy with the model’s results, lets publish the model as a web service to that you can access it from anywhere.

Save the Trained Models

Right-Click on the Linear Regression and Bayesian Trained Model components and select “Save as Trained Model”. We are saving the model to be reused by other programs.

The saved models should show up in the Trained Models list

Now you can use these trained models with live data within your applications.

Prepare Model for Web Service

To prepare your model for two web services, SAVE AS the original experiment two times as:

1) Auto Prediction Linear Regression

2) Auto Prediction Bayesian Linear Regression

Next replace the Linear Regression, Bayesian Linear Regression and their associated Train Model components with our saved models as illustrated below. Run an visualize both the Evaluation Model results.

Evaluation for Linear Regression

Evaluation Results for Bayesian Linear Regression

Note: Typically, you would point both the regressions to the same evaluate model, but the constraint for the Evaluate model to compare both regressions is that they must have the same learner type. Bayesian is a more probabilistic method than linear fit, thus the error.

I did this on purpose for you to understand the capabilities and constraints of AzureML. But, you could instead use Poisson Regression model and evaluate it with Linear Regression.

You can now Visualize the Evaluate Model to compare the Coefficient of Determination.

You an also see the liner fits and prediction performance by visualizing the Score Model.

For the web service, let’s publish the Bayesian Linear Regression because the R2 value looks consistently better than the Linear Regression.

One thing to not forget here is that you can always go back, test the results and publish only when you are satisfied.

Rick-Click the Score Model input endpoint for Bayesian Linear Regression and select “Set as Publish Input”. Next, Right-Click the output endpoint of the Score Model and select “Set as Publish Output”. These two settings specify the data input and output for the web service calls. The final model should look like the figure below.

Finally, after running the model once again, the Publish Web Service button will be enabled. Click on it to proceed with web service publishing to a staging environment.

Testing the Webservice

On the Dashboard, you should see the API Key, and Test link for testing the web service. You can enter the car details for price prediction.

Input Data:  audi sedan 99.8 four 109 102 5500 30 13950

Predicted Price: 12732

Publish Webservice to Production

By default, the web service is created in a staging environment. If you are happy with the prediction, let’s move ahead with publishing the web service to production.

Click “Mark as Ready for Production” link in the Production status section.

Next, in the settings section, select “YES” and click Save.

Next, go back to the Dashboard and click “Deploy to Production” to go to the AzureML webservices page

Click on Add to create the web service in the region of your choice.

The web service gets deployed in seconds (if not minutes). Click on the web service and you will be navigated to the web service management dashboard where you will be able to see the REQUEST/RESPONSE as well as BATCH EXECUTION endpoints for the web service.

Click on the REQUEST/RESPONSE link to go to the endpoint documentation page. On this page you will find the client signatures for C#, R, and Python, and also the input JSON request.

Sample JSON Request

{
  "Id": "score00001",
  "Instance": {
    "FeatureVector": {
      "make": "0",
      "body-style": "0",
      "wheel-base": "0",
      "num-of-cylinders": "0",
      "engine-size": "0",
      "horsepower": "0",
      "peak-rpm": "0",
      "highway-mpg": "0",
      "price": "0"
    },
    "GlobalParameters": {}
  }
} 

C# Client Code

// This code requires the Nuget package Microsoft.AspNet.WebApi.Client to be installed. // Instructions for doing this in Visual Studio: // Tools -> Nuget Package Manager -> Package Manager Console // Install-Package Microsoft.AspNet.WebApi.Client using System; using System.Collections.Generic; using System.IO; using System.Net.Http; using System.Net.Http.Formatting; using System.Net.Http.Headers; using System.Text; using System.Threading.Tasks; namespace CallRequestResponseService { public class ScoreData { public Dictionary<string, string> FeatureVector { get; set; } public Dictionary<string, string> GlobalParameters { get; set; } } public class ScoreRequest { public string Id { get; set; } public ScoreData Instance { get; set; } } class Program { static void Main(string[] args) { InvokeRequestResponseService().Wait(); } static async Task InvokeRequestResponseService() { using (var client = new HttpClient()) { ScoreData scoreData = new ScoreData() { FeatureVector = new Dictionary<string, string>() { { "make", "0" }, { "body-style", "0" }, { "wheel-base", "0" }, { "num-of-cylinders", "0" }, { "engine-size", "0" }, { "horsepower", "0" }, { "peak-rpm", "0" }, { "highway-mpg", "0" }, { "price", "0" }, }, GlobalParameters = new Dictionary<string, string>() { } }; ScoreRequest scoreRequest = new ScoreRequest() { Id = "score00001", Instance = scoreData }; const string apiKey = "abc123"; // Replace this with the API key for the web service client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", apiKey); client.BaseAddress = new Uri("https://ussouthcentral.services.azureml.net/xxxxb62/score"); HttpResponseMessage response = await client.PostAsJsonAsync("", scoreRequest); if (response.IsSuccessStatusCode) { string result = await response.Content.ReadAsStringAsync(); Console.WriteLine("Result: {0}", result); } else { Console.WriteLine("Failed with status code: {0}", response.StatusCode); } } } } }

With the C# Client and REST interface, you can call the web service from any PC, Mobile, and Web clients.

Let’s build a Mobile App

With the web service running in Azure, there is no reason not to build a mobile app user interface. I just quickly put together a Windows Universal App to predict car prices. You can find the source code for the app here.

Conclusion

AzureML is a new and highly productive tool for Machine Learning. It may be the only tool that lets you publish a machine learning web service directly from your design environment. Machine Learning is a vast topic and Linear Regression models discussed in this article only scratches the surface of the topic. In this article, I went over a stale dataset to showcase AzureML as a predictive analytics tool. You can apply the same procedures and components for Classification and Clustering models. Finally, my goal was in writing about Applied Machine Learning. I am not a Data Scientist, but now with all the productive tools, I feel that I can put to work some of the great algorithms that scientists have already invented.

Some more Datasets you can play around with

1. Daily and Sports Activities Data Set link
2. Farm Ads Data Set link
3. Arcene Data Set link
4. Bag of Words Data Set link

Try out AzureML !

References

 https://www.mathworks.com/help/stats/multiple-linear-regression-1.html?refresh=true

https://horicky.blogspot.com/2012/05/predictive-analytics-generalized-linear.html

https://horicky.blogspot.com/2012/05/predictive-analytics-overview-and-data.html

https://101.datascience.community/2013/03/07/9-problems-with-real-world-regression/

https://horicky.blogspot.com/2012/05/predictive-analytics-overview-and-data.html

 https://blogs.msdn.com/b/continuous_learning/

https://stickywinner.com/

By Tejaswi Redkar Windows Azure Web Sites: Building Web Apps at a Rapid Pace (1st Edition)