filter (DataFrame)

Filters rows using the given condition.

Syntax

filter(condition: Union[Column, str])

Parameters

Parameter	Type	Description
condition	Column or str	A Column of BooleanType or a string of SQL expressions.

Returns

DataFrame: A new DataFrame with rows that satisfy the condition.

Examples

df = spark.createDataFrame([
    (2, "Alice", "Math"), (5, "Bob", "Physics"), (7, "Charlie", "Chemistry")],
    schema=["age", "name", "subject"])

df.filter(df.age > 3).show()
# +---+-------+---------+
# |age|   name|  subject|
# +---+-------+---------+
# |  5|    Bob|  Physics|
# |  7|Charlie|Chemistry|
# +---+-------+---------+

df.where(df.age == 2).show()
# +---+-----+-------+
# |age| name|subject|
# +---+-----+-------+
# |  2|Alice|   Math|
# +---+-----+-------+

df.filter("age > 3").show()
# +---+-------+---------+
# |age|   name|  subject|
# +---+-------+---------+
# |  5|    Bob|  Physics|
# |  7|Charlie|Chemistry|
# +---+-------+---------+

df.filter((df.age > 3) & (df.subject == "Physics")).show()
# +---+----+-------+
# |age|name|subject|
# +---+----+-------+
# |  5| Bob|Physics|
# +---+----+-------+