config (SparkSession.Builder)

Sets a config option. Options set using this method are automatically propagated to both SparkConf and SparkSession's own configuration.

Syntax

config(key=None, value=None, conf=None, *, map=None)

Parameters

Parameter	Type	Description
key	str, optional	A key name string for a configuration property.
value	str, optional	A value for the configuration property.
conf	SparkConf, optional	An instance of SparkConf.
map	dict, optional	A dictionary of configurations to set.

Returns

SparkSession.Builder

Examples

# For an existing SparkConf, use the conf parameter.
from pyspark.conf import SparkConf
conf = SparkConf().setAppName("example").setMaster("local")
SparkSession.builder.config(conf=conf)

# For a (key, value) pair, you can omit the parameter names.
SparkSession.builder.config("spark.some.config.option", "some-value")

# Set multiple configurations.
SparkSession.builder.config(
    "spark.some.config.number", 123).config("spark.some.config.float", 0.123)

# Set multiple configurations using a dictionary.
SparkSession.builder.config(
    map={"spark.some.config.number": 123, "spark.some.config.float": 0.123})