Perform incremental processing with spark structured streaming
Intermediate
Data Engineer
Developer
Data Scientist
Data Analyst
Azure Databricks
You explore different features and tools to help you understand and work with incremental processing with spark structured streaming.
Learning objectives
At the end of this module, you're able to:
- Understand Spark structured streaming.
- Some techniques to optimize structured streaming.
- How to handle late arriving or out of order events.
- How to set up real-time-sources for incremental processing.
Prerequisites
- Ability to navigate the Azure portal.
- Understanding of the Azure Databricks workspace.
- Experience with Spark languages and Notebooks.