SynapseSparkJobActivityTypeProperties Class

Package:

com.azure.resourcemanager.datafactory.fluent.models

Maven Artifact:

com.azure.resourcemanager:azure-resourcemanager-datafactory:1.3.0-beta.1

• java.lang.Object
• • com.azure.resourcemanager.datafactory.fluent.models.SynapseSparkJobActivityTypeProperties

Implements

JsonSerializable<SynapseSparkJobActivityTypeProperties>

public final class SynapseSparkJobActivityTypeProperties
implements JsonSerializable<SynapseSparkJobActivityTypeProperties>

Execute spark job activity properties.

Constructor Summary

Constructor	Description
SynapseSparkJobActivityTypeProperties()	Creates an instance of SynapseSparkJobActivityTypeProperties class.

Method Summary

Modifier and Type	Method and Description
List<Object>	arguments() Get the arguments property: User specified arguments to SynapseSparkJobDefinitionActivity.
Object	className() Get the className property: The fully-qualified identifier or the main class that is in the main definition file, which will override the 'className' of the spark job definition you provide.
Object	conf() Get the conf property: Spark configuration properties, which will override the 'conf' of the spark job definition you provide.
ConfigurationType	configurationType() Get the configurationType property: The type of the spark config.
Object	driverSize() Get the driverSize property: Number of core and memory to be used for driver allocated in the specified Spark pool for the job, which will be used for overriding 'driverCores' and 'driverMemory' of the spark job definition you provide.
Object	executorSize() Get the executorSize property: Number of core and memory to be used for executors allocated in the specified Spark pool for the job, which will be used for overriding 'executorCores' and 'executorMemory' of the spark job definition you provide.
Object	file() Get the file property: The main file used for the job, which will override the 'file' of the spark job definition you provide.
List<Object>	files() Get the files property: (Deprecated.
List<Object>	filesV2() Get the filesV2 property: Additional files used for reference in the main definition file, which will override the 'jars' and 'files' of the spark job definition you provide.
static SynapseSparkJobActivityTypeProperties	fromJson(JsonReader jsonReader) Reads an instance of SynapseSparkJobActivityTypeProperties from the JsonReader.
Object	numExecutors() Get the numExecutors property: Number of executors to launch for this job, which will override the 'numExecutors' of the spark job definition you provide.
List<Object>	pythonCodeReference() Get the pythonCodeReference property: Additional python code files used for reference in the main definition file, which will override the 'pyFiles' of the spark job definition you provide.
Object	scanFolder() Get the scanFolder property: Scanning subfolders from the root folder of the main definition file, these files will be added as reference files.
Map<String,Object>	sparkConfig() Get the sparkConfig property: Spark configuration property.
SynapseSparkJobReference	sparkJob() Get the sparkJob property: Synapse spark job reference.
BigDataPoolParametrizationReference	targetBigDataPool() Get the targetBigDataPool property: The name of the big data pool which will be used to execute the spark batch job, which will override the 'targetBigDataPool' of the spark job definition you provide.
SparkConfigurationParametrizationReference	targetSparkConfiguration() Get the targetSparkConfiguration property: The spark configuration of the spark job.
JsonWriter	toJson(JsonWriter jsonWriter)
SynapseSparkJobActivityTypeProperties	withArguments(List<Object> arguments) Set the arguments property: User specified arguments to SynapseSparkJobDefinitionActivity.
SynapseSparkJobActivityTypeProperties	withClassName(Object className) Set the className property: The fully-qualified identifier or the main class that is in the main definition file, which will override the 'className' of the spark job definition you provide.
SynapseSparkJobActivityTypeProperties	withConf(Object conf) Set the conf property: Spark configuration properties, which will override the 'conf' of the spark job definition you provide.
SynapseSparkJobActivityTypeProperties	withConfigurationType(ConfigurationType configurationType) Set the configurationType property: The type of the spark config.
SynapseSparkJobActivityTypeProperties	withDriverSize(Object driverSize) Set the driverSize property: Number of core and memory to be used for driver allocated in the specified Spark pool for the job, which will be used for overriding 'driverCores' and 'driverMemory' of the spark job definition you provide.
SynapseSparkJobActivityTypeProperties	withExecutorSize(Object executorSize) Set the executorSize property: Number of core and memory to be used for executors allocated in the specified Spark pool for the job, which will be used for overriding 'executorCores' and 'executorMemory' of the spark job definition you provide.
SynapseSparkJobActivityTypeProperties	withFile(Object file) Set the file property: The main file used for the job, which will override the 'file' of the spark job definition you provide.
SynapseSparkJobActivityTypeProperties	withFiles(List<Object> files) Set the files property: (Deprecated.
SynapseSparkJobActivityTypeProperties	withFilesV2(List<Object> filesV2) Set the filesV2 property: Additional files used for reference in the main definition file, which will override the 'jars' and 'files' of the spark job definition you provide.
SynapseSparkJobActivityTypeProperties	withNumExecutors(Object numExecutors) Set the numExecutors property: Number of executors to launch for this job, which will override the 'numExecutors' of the spark job definition you provide.
SynapseSparkJobActivityTypeProperties	withPythonCodeReference(List<Object> pythonCodeReference) Set the pythonCodeReference property: Additional python code files used for reference in the main definition file, which will override the 'pyFiles' of the spark job definition you provide.
SynapseSparkJobActivityTypeProperties	withScanFolder(Object scanFolder) Set the scanFolder property: Scanning subfolders from the root folder of the main definition file, these files will be added as reference files.
SynapseSparkJobActivityTypeProperties	withSparkConfig(Map<String,Object> sparkConfig) Set the sparkConfig property: Spark configuration property.
SynapseSparkJobActivityTypeProperties	withSparkJob(SynapseSparkJobReference sparkJob) Set the sparkJob property: Synapse spark job reference.
SynapseSparkJobActivityTypeProperties	withTargetBigDataPool(BigDataPoolParametrizationReference targetBigDataPool) Set the targetBigDataPool property: The name of the big data pool which will be used to execute the spark batch job, which will override the 'targetBigDataPool' of the spark job definition you provide.
SynapseSparkJobActivityTypeProperties	withTargetSparkConfiguration(SparkConfigurationParametrizationReference targetSparkConfiguration) Set the targetSparkConfiguration property: The spark configuration of the spark job.

Methods inherited from java.lang.Object

clone equals finalize getClass hashCode notify notifyAll toString wait wait wait

Constructor Details

SynapseSparkJobActivityTypeProperties

public SynapseSparkJobActivityTypeProperties()

Creates an instance of SynapseSparkJobActivityTypeProperties class.

Method Details

arguments

public List<Object> arguments()

Get the arguments property: User specified arguments to SynapseSparkJobDefinitionActivity.

Returns:

the arguments value.

className

public Object className()

Get the className property: The fully-qualified identifier or the main class that is in the main definition file, which will override the 'className' of the spark job definition you provide. Type: string (or Expression with resultType string).

Returns:

the className value.

conf

public Object conf()

Get the conf property: Spark configuration properties, which will override the 'conf' of the spark job definition you provide.

Returns:

the conf value.

configurationType

public ConfigurationType configurationType()

Get the configurationType property: The type of the spark config.

Returns:

the configurationType value.

driverSize

public Object driverSize()

Get the driverSize property: Number of core and memory to be used for driver allocated in the specified Spark pool for the job, which will be used for overriding 'driverCores' and 'driverMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).

Returns:

the driverSize value.

executorSize

public Object executorSize()

Get the executorSize property: Number of core and memory to be used for executors allocated in the specified Spark pool for the job, which will be used for overriding 'executorCores' and 'executorMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).

Returns:

the executorSize value.

file

public Object file()

Get the file property: The main file used for the job, which will override the 'file' of the spark job definition you provide. Type: string (or Expression with resultType string).

Returns:

the file value.

files

public List<Object> files()

Get the files property: (Deprecated. Please use pythonCodeReference and filesV2) Additional files used for reference in the main definition file, which will override the 'files' of the spark job definition you provide.

Returns:

the files value.

filesV2

public List<Object> filesV2()

Get the filesV2 property: Additional files used for reference in the main definition file, which will override the 'jars' and 'files' of the spark job definition you provide.

Returns:

the filesV2 value.

fromJson

public static SynapseSparkJobActivityTypeProperties fromJson(JsonReader jsonReader)

Reads an instance of SynapseSparkJobActivityTypeProperties from the JsonReader.

Parameters:

jsonReader - The JsonReader being read.

Returns:

An instance of SynapseSparkJobActivityTypeProperties if the JsonReader was pointing to an instance of it, or null if it was pointing to JSON null.

Throws:

IOException

- If the deserialized JSON object was missing any required properties.

numExecutors

public Object numExecutors()

Get the numExecutors property: Number of executors to launch for this job, which will override the 'numExecutors' of the spark job definition you provide. Type: integer (or Expression with resultType integer).

Returns:

the numExecutors value.

pythonCodeReference

public List<Object> pythonCodeReference()

Get the pythonCodeReference property: Additional python code files used for reference in the main definition file, which will override the 'pyFiles' of the spark job definition you provide.

Returns:

the pythonCodeReference value.

scanFolder

public Object scanFolder()

Get the scanFolder property: Scanning subfolders from the root folder of the main definition file, these files will be added as reference files. The folders named 'jars', 'pyFiles', 'files' or 'archives' will be scanned, and the folders name are case sensitive. Type: boolean (or Expression with resultType boolean).

Returns:

the scanFolder value.

sparkConfig

public Map<String,Object> sparkConfig()

Get the sparkConfig property: Spark configuration property.

Returns:

the sparkConfig value.

sparkJob

public SynapseSparkJobReference sparkJob()

Get the sparkJob property: Synapse spark job reference.

Returns:

the sparkJob value.

targetBigDataPool

public BigDataPoolParametrizationReference targetBigDataPool()

Get the targetBigDataPool property: The name of the big data pool which will be used to execute the spark batch job, which will override the 'targetBigDataPool' of the spark job definition you provide.

Returns:

the targetBigDataPool value.

targetSparkConfiguration

public SparkConfigurationParametrizationReference targetSparkConfiguration()

Get the targetSparkConfiguration property: The spark configuration of the spark job.

Returns:

the targetSparkConfiguration value.

toJson

public JsonWriter toJson(JsonWriter jsonWriter)

Parameters:

jsonWriter

Throws:

IOException

withArguments

public SynapseSparkJobActivityTypeProperties withArguments(List<Object> arguments)

Set the arguments property: User specified arguments to SynapseSparkJobDefinitionActivity.

Parameters:

arguments - the arguments value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withClassName

public SynapseSparkJobActivityTypeProperties withClassName(Object className)

Set the className property: The fully-qualified identifier or the main class that is in the main definition file, which will override the 'className' of the spark job definition you provide. Type: string (or Expression with resultType string).

Parameters:

className - the className value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withConf

public SynapseSparkJobActivityTypeProperties withConf(Object conf)

Set the conf property: Spark configuration properties, which will override the 'conf' of the spark job definition you provide.

Parameters:

conf - the conf value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withConfigurationType

public SynapseSparkJobActivityTypeProperties withConfigurationType(ConfigurationType configurationType)

Set the configurationType property: The type of the spark config.

Parameters:

configurationType - the configurationType value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withDriverSize

public SynapseSparkJobActivityTypeProperties withDriverSize(Object driverSize)

Set the driverSize property: Number of core and memory to be used for driver allocated in the specified Spark pool for the job, which will be used for overriding 'driverCores' and 'driverMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).

Parameters:

driverSize - the driverSize value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withExecutorSize

public SynapseSparkJobActivityTypeProperties withExecutorSize(Object executorSize)

Set the executorSize property: Number of core and memory to be used for executors allocated in the specified Spark pool for the job, which will be used for overriding 'executorCores' and 'executorMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).

Parameters:

executorSize - the executorSize value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withFile

public SynapseSparkJobActivityTypeProperties withFile(Object file)

Set the file property: The main file used for the job, which will override the 'file' of the spark job definition you provide. Type: string (or Expression with resultType string).

Parameters:

file - the file value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withFiles

public SynapseSparkJobActivityTypeProperties withFiles(List<Object> files)

Set the files property: (Deprecated. Please use pythonCodeReference and filesV2) Additional files used for reference in the main definition file, which will override the 'files' of the spark job definition you provide.

Parameters:

files - the files value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withFilesV2

public SynapseSparkJobActivityTypeProperties withFilesV2(List<Object> filesV2)

Set the filesV2 property: Additional files used for reference in the main definition file, which will override the 'jars' and 'files' of the spark job definition you provide.

Parameters:

filesV2 - the filesV2 value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withNumExecutors

public SynapseSparkJobActivityTypeProperties withNumExecutors(Object numExecutors)

Set the numExecutors property: Number of executors to launch for this job, which will override the 'numExecutors' of the spark job definition you provide. Type: integer (or Expression with resultType integer).

Parameters:

numExecutors - the numExecutors value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withPythonCodeReference

public SynapseSparkJobActivityTypeProperties withPythonCodeReference(List<Object> pythonCodeReference)

Set the pythonCodeReference property: Additional python code files used for reference in the main definition file, which will override the 'pyFiles' of the spark job definition you provide.

Parameters:

pythonCodeReference - the pythonCodeReference value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withScanFolder

public SynapseSparkJobActivityTypeProperties withScanFolder(Object scanFolder)

Set the scanFolder property: Scanning subfolders from the root folder of the main definition file, these files will be added as reference files. The folders named 'jars', 'pyFiles', 'files' or 'archives' will be scanned, and the folders name are case sensitive. Type: boolean (or Expression with resultType boolean).

Parameters:

scanFolder - the scanFolder value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withSparkConfig

public SynapseSparkJobActivityTypeProperties withSparkConfig(Map<String,Object> sparkConfig)

Set the sparkConfig property: Spark configuration property.

Parameters:

sparkConfig - the sparkConfig value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withSparkJob

public SynapseSparkJobActivityTypeProperties withSparkJob(SynapseSparkJobReference sparkJob)

Set the sparkJob property: Synapse spark job reference.

Parameters:

sparkJob - the sparkJob value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withTargetBigDataPool

public SynapseSparkJobActivityTypeProperties withTargetBigDataPool(BigDataPoolParametrizationReference targetBigDataPool)

Set the targetBigDataPool property: The name of the big data pool which will be used to execute the spark batch job, which will override the 'targetBigDataPool' of the spark job definition you provide.

Parameters:

targetBigDataPool - the targetBigDataPool value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.

withTargetSparkConfiguration

public SynapseSparkJobActivityTypeProperties withTargetSparkConfiguration(SparkConfigurationParametrizationReference targetSparkConfiguration)

Set the targetSparkConfiguration property: The spark configuration of the spark job.

Parameters:

targetSparkConfiguration - the targetSparkConfiguration value to set.

Returns:

the SynapseSparkJobActivityTypeProperties object itself.