Kaganapan
Mar 31, 11 PM - Abr 2, 11 PM
Ang pinakamalaking Tela, Power BI, at SQL learning event. Marso 31 – Abril 2. Gamitin ang code FABINSIDER upang makatipid ng $ 400.
Magparehistro naWhat is an Apache Spark job definition?
An Apache Spark job definition is a Microsoft Fabric code item that allows you to submit batch/streaming jobs to Spark clusters. By uploading the binary files from the compilation output of different languages (for example, .jar from Java), you can apply different transformation logic to the data hosted on a lakehouse. Besides the binary file, you can further customize the behavior of the job by uploading more libraries and command line arguments.
To run a Spark job definition, you must have at least one lakehouse associated with it. This default lakehouse context serves as the default file system for Spark runtime. For any Spark code using a relative path to read/write data, the data is served from the default lakehouse.
Tip
To run a Spark job definition item, you must have a main definition file and default lakehouse context. If you don't have a lakehouse, create one by following the steps in Create a lakehouse.
Mga karagdagang mapagkukunan
Pagsasanay
Module
Use Apache Spark in Microsoft Fabric - Training
Apache Spark is a core technology for large-scale data analytics. Microsoft Fabric provides support for Spark clusters, enabling you to analyze and process data at scale.
Sertipikasyon
Microsoft Certified: Fabric Data Engineer Associate - Certifications
As a Fabric Data Engineer, you should have subject matter expertise with data loading patterns, data architectures, and orchestration processes.