map_from_entries

Transforms an array of key-value pair entries (structs with two fields) into a map. The first field of each entry is used as the key and the second field as the value in the resulting map column.

Syntax

from pyspark.sql import functions as sf

sf.map_from_entries(col)

Parameters

Parameter	Type	Description
col	pyspark.sql.Column or str	Name of column or expression

Returns

pyspark.sql.Column: A map created from the given array of entries.

Examples

Example 1: Basic usage of map_from_entries

from pyspark.sql import functions as sf
df = spark.sql("SELECT array(struct(1, 'a'), struct(2, 'b')) as data")
df.select(sf.map_from_entries(df.data)).show()

+----------------------+
|map_from_entries(data)|
+----------------------+
|      {1 -> a, 2 -> b}|
+----------------------+

Example 2: map_from_entries with null values

from pyspark.sql import functions as sf
df = spark.sql("SELECT array(struct(1, null), struct(2, 'b')) as data")
df.select(sf.map_from_entries(df.data)).show()

+----------------------+
|map_from_entries(data)|
+----------------------+
|   {1 -> NULL, 2 -> b}|
+----------------------+

Example 3: map_from_entries with a DataFrame

from pyspark.sql import Row, functions as sf
df = spark.createDataFrame([([Row(1, "a"), Row(2, "b")],), ([Row(3, "c")],)], ['data'])
df.select(sf.map_from_entries(df.data)).show()

+----------------------+
|map_from_entries(data)|
+----------------------+
|      {1 -> a, 2 -> b}|
|              {3 -> c}|
+----------------------+

Example 4: map_from_entries with empty array

from pyspark.sql import functions as sf
from pyspark.sql.types import ArrayType, StringType, IntegerType, StructType, StructField
schema = StructType([
  StructField("data", ArrayType(
    StructType([
      StructField("key", IntegerType()),
      StructField("value", StringType())
    ])
  ), True)
])
df = spark.createDataFrame([([],)], schema=schema)
df.select(sf.map_from_entries(df.data)).show()

+----------------------+
|map_from_entries(data)|
+----------------------+
|                    {}|
+----------------------+