Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Returns whether a predicate holds for one or more elements in the array. Supports Spark Connect.
For the corresponding Databricks SQL function, see exists function.
Syntax
from pyspark.databricks.sql import functions as dbf
dbf.exists(col=<col>, f=<f>)
Parameters
| Parameter | Type | Description |
|---|---|---|
col |
pyspark.sql.Column or str |
Name of column or expression. |
f |
function |
A function that returns the Boolean expression. |
Returns
pyspark.sql.Column: True if "any" element of an array evaluates to True when passed as an argument to given function and False otherwise.
Examples
from pyspark.databricks.sql import functions as dbf
df = spark.createDataFrame([(1, [1, 2, 3, 4]), (2, [3, -1, 0])],("key", "values"))
df.select(dbf.exists("values", lambda x: x < 0).alias("any_negative")).show()
+------------+
|any_negative|
+------------+
| false|
| true|
+------------+