SparkJobProperties interface

Package:

@azure/synapse-artifacts

The properties of the Spark job.

Properties

archives	Archives to be used in this job.
args	Command line arguments for the application.
className	Main class for Java/Scala application.
conf	Spark configuration properties.
driverCores	Number of cores to use for the driver.
driverMemory	Amount of memory to use for the driver process.
executorCores	Number of cores to use for each executor.
executorMemory	Amount of memory to use per executor process.
file	File containing the application to execute.
files	files to be used in this job.
jars	Jars to be used in this job.
name	The name of the job.
numExecutors	Number of executors to launch for this job.

Property Details

archives

Archives to be used in this job.

archives?: string[]

Property Value

string[]

args

Command line arguments for the application.

args?: string[]

Property Value

string[]

className

Main class for Java/Scala application.

className?: string

Property Value

string

conf

Spark configuration properties.

conf?: any

Property Value

any

driverCores

Number of cores to use for the driver.

driverCores: number

Property Value

number

driverMemory

Amount of memory to use for the driver process.

driverMemory: string

Property Value

string

executorCores

Number of cores to use for each executor.

executorCores: number

Property Value

number

executorMemory

Amount of memory to use per executor process.

executorMemory: string

Property Value

string

file

File containing the application to execute.

file: string

Property Value

string

files

files to be used in this job.

files?: string[]

Property Value

string[]

jars

Jars to be used in this job.

jars?: string[]

Property Value

string[]

name

The name of the job.

name?: string

Property Value

string

numExecutors

Number of executors to launch for this job.

numExecutors: number

Property Value

number