median

Returns the median of the values in a group.

Syntax

from pyspark.sql import functions as sf

sf.median(col)

Parameters

Parameter	Type	Description
col	pyspark.sql.Column or column name	Target column to compute on.

Returns

pyspark.sql.Column: the median of the values in a group.

Examples

from pyspark.sql import functions as sf
df = spark.createDataFrame([
    ("Java", 2012, 20000), ("dotNET", 2012, 5000),
    ("Java", 2012, 22000), ("dotNET", 2012, 10000),
    ("dotNET", 2013, 48000), ("Java", 2013, 30000)],
    schema=("course", "year", "earnings"))
df.groupby("course").agg(sf.median("earnings")).show()

+------+----------------+
|course|median(earnings)|
+------+----------------+
|  Java|         22000.0|
|dotNET|         10000.0|
+------+----------------+