Use Spark Notebooks in an Azure Synapse Pipeline

Module
7 Units

Intermediate

Data Engineer

Azure Synapse Analytics

Apache Spark provides data engineers with a scalable, distributed data processing platform, which can be integrated into an Azure Synapse Analytics pipeline.

Learning objectives

In this module, you will learn how to:

Describe notebook and pipeline integration.
Use a Synapse notebook activity in a pipeline.
Use parameters with a notebook activity.

Prerequisites

Before starting this module, you should have experience of using Apache Spark in Azure Synapse Analytics. Consider completing the Analyze data with Apache Spark in Azure Synapse Analytics module first:

Introduction min
Understand Synapse Notebooks and Pipelines min
Use a Synapse notebook activity in a pipeline min
Use parameters in a notebook min
Exercise - Use an Apache Spark notebook in a pipeline min
Knowledge check min
Summary min