Episode
Microsoft Fabric Learn Together Ep08: Ingest data with Spark and Microsoft Fabric notebooks
with Johan Ludvig Brattås, Heini Ilmarinen, Marcel Magalhães, Olivier Van Steenlandt
Discover how to use Apache Spark and Python for data ingestion into a Microsoft Fabric lakehouse. Fabric notebooks provide a scalable and systematic solution.
Learning objectives
- Ingest external data to Fabric lakehouses using Spark
- Configure external source authentication and optimization
- Load data into lakehouse as files or as Delta tables
Chapters
- 00:00 - Introduction
- 05:48 - Learning objectives
- 06:21 - Context in Fabric
- 10:31 - What is Spark?
- 14:54 - An introduction to notebooks
- 16:42 - Explore Fabric notebooks
- 22:28 - What is a lakehouse?
- 25:44 - Fabric lakehouse vs Data warehouse
- 28:47 - Write data into a lakehouse
- 30:57 - Write to a Delta table
- 34:13 - Consider uses for ingested data
- 42:01 - Exercise - Ingest data with Spark and Microsoft Fabric notebooks
- 01:04:51 - Knowledge check
- 01:13:24 - Summary
Recommended resources
Related episodes
- Full series: Learn Live: Learn Together Microsoft Fabric
Connect
- Johan Ludvig Brattås | LinkedIn: /in/johanludvig
- Heini Ilmarinen | Twitter: @HeiniIlmarinen | LinkedIn: /in/heini-ilmarinen
- Marcel Magalhães | LinkedIn: /in/marcelsmagalhaes
- Olivier Van Steenlandt | LinkedIn: /in/oliviervansteenlandt
Discover how to use Apache Spark and Python for data ingestion into a Microsoft Fabric lakehouse. Fabric notebooks provide a scalable and systematic solution.
Learning objectives
- Ingest external data to Fabric lakehouses using Spark
- Configure external source authentication and optimization
- Load data into lakehouse as files or as Delta tables
Chapters
- 00:00 - Introduction
- 05:48 - Learning objectives
- 06:21 - Context in Fabric
- 10:31 - What is Spark?
- 14:54 - An introduction to notebooks
- 16:42 - Explore Fabric notebooks
- 22:28 - What is a lakehouse?
- 25:44 - Fabric lakehouse vs Data warehouse
- 28:47 - Write data into a lakehouse
- 30:57 - Write to a Delta table
- 34:13 - Consider uses for ingested data
- 42:01 - Exercise - Ingest data with Spark and Microsoft Fabric notebooks
- 01:04:51 - Knowledge check
- 01:13:24 - Summary
Recommended resources
Related episodes
- Full series: Learn Live: Learn Together Microsoft Fabric
Connect
- Johan Ludvig Brattås | LinkedIn: /in/johanludvig
- Heini Ilmarinen | Twitter: @HeiniIlmarinen | LinkedIn: /in/heini-ilmarinen
- Marcel Magalhães | LinkedIn: /in/marcelsmagalhaes
- Olivier Van Steenlandt | LinkedIn: /in/oliviervansteenlandt
Video URL
HTML iframe
Intermediate
Data Analyst
Data Engineer
Data Scientist
Microsoft Fabric
Have feedback? Submit an issue here.