Explore best practices for module 11

  • 7 minutes

DP-203: Module 11 Create a stream processing solution with Event Hubs and Azure Databricks

Module 11 Overview

As you begin to teach this module, get familiar with what the students will learn during the module. In this module students will learn how to create a stream processing solution with Event Hubs and Azure Databricks, how to stream data from a file, write it out to the distributed file system, and connect to Event Hubs to read and write streams using Spark Structured Streaming. They will also learn when to use Spark Streaming versus Stream Analytics.

This module consists of four lessons:

  • Lesson 1 – Understand the key features and uses of Structured Streaming
  • Lesson 2 – Stream data from a file and write it out to a distributed file system and connect to Event Hubs to read and write streams
  • Lesson 3 – Use sliding windows to aggregate over chunks of data rather than all data
  • Lesson 4 – Apply watermarking

Module 11 Tips and Tricks

  • The concepts for this module are much the same as the previous module. It only has replaced Stream Analytics with Databricks
  • Spark is a much larger engine that Stream Analytics, so it allows you to join to much more static/reference data

Module 11 Demo

  • Optional: Demonstrate the setup that is done in the labs

Module 11 Lab/Exercise

  • Create a stream processing solution with Event Hubs and Azure Databricks (~45 min)