Piezīmes
Lai piekļūtu šai lapai, ir nepieciešama autorizācija. Varat mēģināt pierakstīties vai mainīt direktorijus.
Lai piekļūtu šai lapai, ir nepieciešama autorizācija. Varat mēģināt mainīt direktorijus.
Collection: Merges two given maps into a single map by applying a function to the key-value pairs. Supports Spark Connect.
For the corresponding Databricks SQL function, see map_zip_with function.
Syntax
from pyspark.databricks.sql import functions as dbf
dbf.map_zip_with(col1=<col1>, col2=<col2>, f=<f>)
Parameters
| Parameter | Type | Description |
|---|---|---|
col1 |
pyspark.sql.Column or str |
The name of the first column or a column expression representing the first map. |
col2 |
pyspark.sql.Column or str |
The name of the second column or a column expression representing the second map. |
f |
function |
A ternary function that defines how to merge the values from the two maps. This function should return a column that will be used as the value in the resulting map. |
Returns
pyspark.sql.Column: A new map column where each key-value pair is the result of applying the function to the corresponding key-value pairs in the input maps.
Examples
Example 1: Merging two maps with a simple function
from pyspark.databricks.sql import functions as dbf
df = spark.createDataFrame([
(1, {"A": 1, "B": 2}, {"A": 3, "B": 4})],
("id", "map1", "map2"))
row = df.select(
dbf.map_zip_with("map1", "map2", lambda _, v1, v2: v1 + v2).alias("updated_data")
).head()
sorted(row["updated_data"].items())
[('A', 4), ('B', 6)]
Example 2: Merging two maps with mismatched keys
from pyspark.databricks.sql import functions as dbf
df = spark.createDataFrame([
(1, {"A": 1, "B": 2}, {"B": 3, "C": 4})],
("id", "map1", "map2"))
row = df.select(
dbf.map_zip_with("map1", "map2",
lambda _, v1, v2: dbf.when(v2.isNull(), v1).otherwise(v1 + v2)
).alias("updated_data")
).head()
sorted(row["updated_data"].items())
[('A', 1), ('B', 5), ('C', None)]