Builder Class
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
The entry point to programming Spark with the Dataset and DataFrame API.
public sealed class Buildertype Builder = classPublic NotInheritable Class Builder- Inheritance
-
Builder
Methods
| AppName(String) |
Sets a name for the application, which will be shown in the Spark web UI. If no application name is set, a randomly generated name will be used. |
| Config(SparkConf) |
Sets a list of config options based on the given SparkConf |
| Config(String, Boolean) |
Sets a config option. Options set using this method are automatically propagated to both SparkConf and SparkSession's own configuration. |
| Config(String, Double) |
Sets a config option. Options set using this method are automatically propagated to both SparkConf and SparkSession's own configuration. |
| Config(String, Int64) |
Sets a config option. Options set using this method are automatically propagated to both SparkConf and SparkSession's own configuration. |
| Config(String, String) |
Sets a config option. Options set using this method are automatically propagated to both SparkConf and SparkSession's own configuration. |
| EnableHiveSupport() |
Enables Hive support, including connectivity to a persistent Hive metastore, support for Hive serdes, and Hive user-defined functions. |
| GetOrCreate() |
Gets an existing [[SparkSession]] or, if there is no existing one, creates a new one based on the options set in this builder. |
| Master(String) |
Sets the Spark master URL to connect to, such as "local" to run locally, "local[4]" to run locally with 4 cores, or "spark://master:7077" to run on a Spark standalone cluster. |