Rediger

Contactless IoT interfaces with Azure intelligent edge

Azure AI services
Azure IoT Edge
Azure IoT Hub
Azure Storage
Azure Machine Learning

Solution ideas

This article is a solution idea. If you'd like us to expand the content with more information, such as potential use cases, alternative services, implementation considerations, or pricing guidance, let us know by providing GitHub feedback.

Contactless business is the new normal. The world has become more aware of, and endangered by, surfaces that many people touch every day. Contactless interfaces reduce or eliminate physical touchpoints like traffic light buttons, touch screens, door handles, and elevator controls by creating touch-free experiences that are both safe and pleasing for users.

Avanade and the Microsoft COVID-19 task force partnered to develop contactless interfaces using the Azure intelligent edge platform. This solution combines intelligent and perceptive Internet of Things (IoT) edge devices with the storage, computing, artificial intelligence (AI), and machine learning (ML) capabilities of the Azure cloud.

IoT Edge devices can quickly recognize and respond to speech, image, gesture, or multi-modal input by using onboard processing. Azure IoT Hub in the cloud controls the devices and connects them to Azure resources. Azure Cognitive Services and Machine Learning continually retrain and update models to improve interface accuracy and performance.

Potential use cases

  • Turn public touchpoints to touch-free interfaces for uses like building and room access, elevator controls, retail and vending machine sales, and transit and traffic signaling.
  • Use voice or other contactless controls with typical mobile endpoints like phones.

Architecture

Architecture diagram: Contactless interfaces and other IoT edge devices used as part of an Azure intelligent cloud solution.

Download a Visio file of this architecture.

  1. End points like microphones, cameras, and touchscreens collect data.
  2. Onboard voice processing units on the IoT Edge devices apply cognitive skill sets and updated machine learning models to local data.
  3. The IoT hub in the Azure cloud controls and communicates with the edge devices, receiving data and sending updated models.
  4. Azure Storage stores the uploaded data.
  5. Azure Machine Learning uses the data to retrain its AI models.
  6. IoT Hub pushes updated machine learning models to the edge devices.

Components

  • Azure IoT Edge service deploys cloud workloads to run on IoT Edge devices via standard containers. Modules can run AI, other Azure and third-party services, or your own business logic. IoT Edge intelligent devices can respond quickly and offline, and limit costs by preprocessing and sending only necessary data to the cloud.
  • Azure IoT Hub provides a cloud-hosted backend to connect virtually any IoT device to Azure services. IoT Hub enables highly secure and reliable bi-directional communication, management, and provisioning for IoT Edge devices.
  • Azure Storage provides flexible, scalable, secure storage in the Azure cloud. The current solution uses block blobs to store unstructured data, page blobs to read and write random small data segments, and file storage for file shares.
  • Azure Cognitive Services is a family of AI services and cognitive APIs that help build intelligent apps. For example, voice control can use the Speech to Text and Speaker Recognition services. Extending the solution to image or facial recognition could use Computer Vision, Custom Vision, and Facial Recognition.
  • Machine Learning (ML) uses algorithms to improve machine predictions or decisions automatically through experience. Machine learning algorithms build and continually train mathematical models. Azure Machine Learning lets you build, train, deploy, track, and manage ML models at cloud scale.

Next steps