coalesce

Returns the first column that is not null. Supports Spark Connect.

For the corresponding Databricks SQL function, see coalesce function.

Syntax

from pyspark.sql import functions as dbf

dbf.coalesce(*cols)

Parameters

Parameter	Type	Description
cols	pyspark.sql.Column or str	List of columns to work on.

Returns

pyspark.sql.Column: value of the first column that is not null.

Examples

from pyspark.sql import functions as dbf
df = spark.createDataFrame([(None, None), (1, None), (None, 2)], ("a", "b"))
df.select('*', dbf.coalesce("a", df["b"])).show()

+----+----+--------------+
|   a|   b|coalesce(a, b)|
+----+----+--------------+
|NULL|NULL|          NULL|
|   1|NULL|             1|
|NULL|   2|             2|
+----+----+--------------+