Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Returns col1 if it is not NaN, or col2 if col1 is NaN. Both inputs should be floating point columns (DoubleType or FloatType). Supports Spark Connect.
For the corresponding Databricks SQL function, see nanvl function.
Syntax
from pyspark.databricks.sql import functions as dbf
dbf.nanvl(col1=<col1>, col2=<col2>)
Parameters
| Parameter | Type | Description |
|---|---|---|
col1 |
pyspark.sql.Column or str |
First column to check. |
col2 |
pyspark.sql.Column or str |
Second column to return if first is NaN. |
Returns
pyspark.sql.Column: value from first column or second if first is NaN .
Examples
from pyspark.databricks.sql import functions as dbf
df = spark.createDataFrame([(1.0, float('nan')), (float('nan'), 2.0)], ("a", "b"))
df.select("*", dbf.nanvl("a", "b"), dbf.nanvl(df.a, df.b)).show()
+---+---+-----------+-----------+
| a| b|nanvl(a, b)|nanvl(a, b)|
+---+---+-----------+-----------+
|1.0|NaN| 1.0| 1.0|
|NaN|2.0| 2.0| 2.0|
+---+---+-----------+-----------+