Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Gets an existing SparkSession or, if there is no existing one, creates a new one based on the options set in this builder.
This method first checks whether there is a valid global default SparkSession and, if so, returns that one. If no valid global default SparkSession exists, the method creates a new SparkSession and assigns it as the global default. When an existing SparkSession is returned, the config options specified in this builder are applied to it. To create a new session without checking, use create.
Syntax
getOrCreate()
Returns
SparkSession
Examples
s1 = SparkSession.builder.config("k1", "v1").getOrCreate()
s1.conf.get("k1") == "v1"
# True
# The configuration of the SparkSession can be changed afterwards.
s1.conf.set("k1", "v1_new")
s1.conf.get("k1") == "v1_new"
# True
# When an existing SparkSession is returned, the config options specified
# in this builder are applied to the existing SparkSession.
s2 = SparkSession.builder.config("k2", "v2").getOrCreate()
s1.conf.get("k1") == s2.conf.get("k1") == "v1_new"
# True
s1.conf.get("k2") == s2.conf.get("k2") == "v2"
# True