הערה
הגישה לדף זה מחייבת הרשאה. באפשרותך לנסות להיכנס או לשנות מדריכי כתובות.
הגישה לדף זה מחייבת הרשאה. באפשרותך לנסות לשנות מדריכי כתובות.
Converts a column containing nested inputs (array/map/struct) into a variants where maps and structs are converted to variant objects which are unordered unlike SQL structs. Input maps can only have string keys.
Syntax
from pyspark.sql import functions as sf
sf.to_variant_object(col)
Parameters
| Parameter | Type | Description |
|---|---|---|
col |
pyspark.sql.Column or str |
A column with a nested schema or column name. |
Returns
pyspark.sql.Column: a new column of VariantType.
Examples
Example 1: Converting an array containing a nested struct into a variant
from pyspark.sql import functions as sf
from pyspark.sql.types import ArrayType, StructType, StructField, StringType, MapType
schema = StructType([
StructField("i", StringType(), True),
StructField("v", ArrayType(StructType([
StructField("a", MapType(StringType(), StringType()), True)
]), True))
])
data = [("1", [{"a": {"b": 2}}])]
df = spark.createDataFrame(data, schema)
df.select(sf.to_variant_object(df.v))
DataFrame[to_variant_object(v): variant]
df.select(sf.to_variant_object(df.v)).show(truncate=False)
+--------------------+
|to_variant_object(v)|
+--------------------+
|[{"a":{"b":"2"}}] |
+--------------------+