toPandas

Returns the contents of this DataFrame as Pandas pandas.DataFrame.

Syntax

toPandas()

Returns

pandas.DataFrame

Notes

This method should only be used if the resulting Pandas pandas.DataFrame is expected to be small, as all the data is loaded into the driver's memory.

Usage with spark.sql.execution.arrow.pyspark.enabled=True is experimental.

This is only available if Pandas is installed and available.

Examples

df = spark.createDataFrame([(2, "Alice"), (5, "Bob")], schema=["age", "name"])
df.toPandas()
#    age   name
# 0    2  Alice
# 1    5    Bob