Lakeflow Spark Declarative Pipelines tutorials

These tutorials give you hands-on experience building pipelines with Lakeflow Spark Declarative Pipelines (SDP), from your first pipeline to ETL with CDC, geospatial ingestion, and source-controlled bundles.

Get started with Lakeflow Spark Declarative Pipelines

Topic	Description
Tutorial: Build an ETL pipeline with Lakeflow Spark Declarative Pipelines	This tutorial shows you how to create and deploy a simple ETL (extract, transform, and load) pipeline for data orchestration and Auto Loader.

Tutorials

Topic	Description
Tutorial: Create your first pipeline using the Lakeflow Pipelines Editor	This tutorial walks you through creating your first pipeline by extending the sample code that is created with a new pipeline.
Tutorial: Build an ETL pipeline using change data capture	This tutorial walks you through the steps to create and deploy an ETL pipeline with change data capture (CDC) using SDP for data orchestration and Auto Loader.
ETL in Databricks SQL	This tutorial walks you through building an incremental ETL pipeline in Databricks SQL using streaming tables, AUTO CDC, and materialized views.
Tutorial: Build a geospatial pipeline with native spatial types	This tutorial walks you through building a geospatial pipeline that ingests GPS data, converts coordinates to native spatial types, and joins against warehouse geofences using SDP and Auto Loader.
Create a source-controlled pipeline	This tutorial shows you how to create a source-controlled pipeline using Declarative Automation Bundles and Git folders.
Convert a pipeline into a bundle project	This tutorial shows you how to convert an existing pipeline into a Declarative Automation Bundles project, to provide easier maintenance and automated deployment to target environments.