Announcing the General Availability of Azure Stream Analytics

  • Article

This post is authored by Joseph Sirosh , Corporate Vice President of Information Management & Machine Learning at Microsoft.

As part of our ongoing commitment to deliver a world-class data platform, I am thrilled to announce the general availability of Azure Stream Analytics, a fully managed cloud service for real-time processing of streaming data. This new service extends the broad portfolio of solutions we offer to help customers unlock deep insights from their data.

Azure Stream Analytics (ASA) makes it easy to set up real-time analytic computations on data flowing in from devices, sensors, web sites, applications and infrastructure systems. It supports a powerful high-level SQL-like language that dramatically simplifies the logic to visualize, alert, or act in near real-time. ASA makes it simpler to build a wide range of Internet-of-Things (IoT) applications such as real-time remote device management, and to monitor and gain analytic insights from connected devices of all types including mobile phones and connected cars.

ASA is a secure, fully managed multi-tenant service that makes it possible to analyze vast amounts of flowing data economically. Important features such as resiliency and check pointing for auto recovery are built in to the platform. Customers can allocate resources based on their scale and performance needs and pay for only the resources they use – and it’s easy to start small and cheap and scale to terabyte-scale scenarios. 

ASA provides a rapid development experience, removing all unnecessary overhead of traditional programming languages such as Java, and allowing stream analytics to be written in a familiar, compact, SQL-like language. For example, computing a moving average on a temporal window takes five lines of code in ASA compared to over hundreds of lines in Java/Storm. Simple configuration settings allow developers to tackle the complexities of managing network latencies from sensors sending data to the cloud, correctly ordering events across thousands of sensors to find correct patterns etc. The stream analysis logic can also be easily tested and debugged within the internet browser itself before deploying it live in the cloud.

ASA has been used in a wide variety of scenarios including smart grid management, predictive maintenance and remote device monitoring. Customers such as Fujitsu, NEC and Aerocrine are currently among the customers using ASA to perform real-time analytics on multiple streams of data from internet-enabled devices.

For example,Fujitsu, a leading Japanese information and communications technology company, is using ASA for environmental monitoring and management in manufacturing. Their solution collects data from various sensors and machines and uses ASA for real-time analytics to accelerate factory-wide optimization by improving product quality, streamlining systems and enhancing functionality while reducing costs. Mr. Hiromitsu Oikawa, Director, Fujitsu Limited, is impressed with the capabilities ASA provides:

“In the past, CEP (Complex Event Processing) in the cloud was not a realistic option, but Azure Stream Analytics showed great performance running on Azure. I found that ASA integrated with Event Hubs provides real-time processing with real-time visualization and it leads to real-time control leveraged by cloud. As a result, ASA will be able to improve QCDSE (Quality, Cost, Delivery, Safety, Environment) alongside other IoT solutions deployed in factories.”

Drawing on its strengths in display technology and biometric solutions including face detection and recognition, NEC has become a leader in intelligent digital signage. ASA enables real-time, insightful analysis of shopper behavior and, thus greater value for their retail customers. NEC looks forward to continuing to expand its biometric solutions which incorporate Microsoft Kinect and Azure services. Such solutions allow customers to quickly transform their business processes in today’s highly competitive retail environment:

“NEC has found that using the Azure IoT Services has enabled us to quickly build compelling intelligent digital signage solutions that meet our customer’s needs and help them transform their business processes,” said Allen Ganz, Director of Business Development, NEC Corporation of America. “Azure Stream Analytics is a great addition to Microsoft’s offerings as it allows us to provide real-time analysis of shopper behavior to our retail customers with minimal development effort.”

 

Aerocrine, based in Sweden, helps millions of asthma sufferers worldwide to diagnose and treat their disease effectively. Their devices are sensitive to changes in the ambient environment and the company uses ASA to remotely monitor them for service and sales:

“We can easily collect near-real-time telemetry data from all our devices in clinics and hospitals worldwide with the Microsoft Azure solution,” said Matt Carlson, Vice President of Global Business Development at Aerocrine. “We can use Event Hubs and Stream Analytics to find out if an instrument has a humidity level that’s too high or too low. We can determine if devices are outside normal limits.”

New in the General Availability release are several added features and innovations. Here are some highlights:

  • Reference Data Refresh: We know that business logic changes over time, so we have added the ability to ensure that reference, or lookup, data can be automatically updated while a job is still running. You can now specify a dynamic path as the location of your reference data file, for example: products/{date}/{time}/product-list.csv

  • Diagnose Feature: To improve the ease of debugging for streaming jobs, we have surfaced a high-level Diagnosis to show the health of inputs and outputs at a glance:


    Clicking on the resource provides detailed diagnostic information, if applicable:

  • Creating New Resource: You can now create new instances of the desired Azure Services within the Add Input and Add Output workflows, without leaving the context of your ASA job. This includes support for SQL Database, Blob Storage, Event Hub, and Table Storage creation.

Managed data services from Microsoft now provide unparalleled opportunities for businesses, developers and IT professionals to extract the most value out of every byte of their data, be it relational or non-relational data, structured or unstructured data, fixed or evolving data models.

I encourage you to try out the newly released Azure Stream Analytics service, and let us know what you think via the Stream Analytics forum.

Joseph
Follow me on Twitter