返回所有给定映射的并集。 对于输入映射中的重复键,处理由 spark.sql.mapKeyDedupPolicy它控制。 默认情况下,它会引发异常。 如果设置为 LAST_WIN,则使用最后一个映射的值。
Syntax
from pyspark.sql import functions as sf
sf.map_concat(*cols)
参数
| 参数 | 类型 | Description |
|---|---|---|
cols |
pyspark.sql.Column 或 str |
列名或列 |
退货
pyspark.sql.Column:来自其他映射的合并条目的映射。
例子
示例 1:map_concat的基本用法
from pyspark.sql import functions as sf
df = spark.sql("SELECT map(1, 'a', 2, 'b') as map1, map(3, 'c') as map2")
df.select(sf.map_concat("map1", "map2")).show(truncate=False)
+------------------------+
|map_concat(map1, map2) |
+------------------------+
|{1 -> a, 2 -> b, 3 -> c}|
+------------------------+
示例 2:具有三个地图的map_concat
from pyspark.sql import functions as sf
df = spark.sql("SELECT map(1, 'a') as map1, map(2, 'b') as map2, map(3, 'c') as map3")
df.select(sf.map_concat("map1", "map2", "map3")).show(truncate=False)
+----------------------------+
|map_concat(map1, map2, map3)|
+----------------------------+
|{1 -> a, 2 -> b, 3 -> c} |
+----------------------------+
示例 3:使用空地图map_concat
from pyspark.sql import functions as sf
df = spark.sql("SELECT map(1, 'a', 2, 'b') as map1, map() as map2")
df.select(sf.map_concat("map1", "map2")).show(truncate=False)
+----------------------+
|map_concat(map1, map2)|
+----------------------+
|{1 -> a, 2 -> b} |
+----------------------+
示例 4:具有 null 值的map_concat
from pyspark.sql import functions as sf
df = spark.sql("SELECT map(1, 'a', 2, 'b') as map1, map(3, null) as map2")
df.select(sf.map_concat("map1", "map2")).show(truncate=False)
+---------------------------+
|map_concat(map1, map2) |
+---------------------------+
|{1 -> a, 2 -> b, 3 -> NULL}|
+---------------------------+