Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Separates a variant object/array into multiple rows containing its fields/elements. Its result schema is struct<pos int, key string, value variant>. pos is the position of the field/element in its parent object/array, and value is the field/element value. key is the field name when exploding a variant object, or is NULL when exploding a variant array. It ignores any input that is not a variant array/object, including SQL NULL, variant null, and any other variant values.
Syntax
spark.tvf.variant_explode(input)
Parameters
| Parameter | Type | Description |
|---|---|---|
input |
pyspark.sql.Column |
Input column of values to explode. |
Examples
Example 1: Using variant_explode with a variant array
from pyspark.sql import functions as sf
spark.tvf.variant_explode(sf.parse_json(sf.lit('["hello", "world"]'))).show()
+---+----+-------+
|pos| key| value|
+---+----+-------+
| 0|NULL|"hello"|
| 1|NULL|"world"|
+---+----+-------+
Example 2: Using variant_explode with a variant object
from pyspark.sql import functions as sf
spark.tvf.variant_explode(sf.parse_json(sf.lit('{"a": true, "b": 3.14}'))).show()
+---+---+-----+
|pos|key|value|
+---+---+-----+
| 0| a| true|
| 1| b| 3.14|
+---+---+-----+
Example 3: Using variant_explode with an empty variant array
from pyspark.sql import functions as sf
spark.tvf.variant_explode(sf.parse_json(sf.lit('[]'))).show()
+---+---+-----+
|pos|key|value|
+---+---+-----+
+---+---+-----+
Example 4: Using variant_explode with an empty variant object
from pyspark.sql import functions as sf
spark.tvf.variant_explode(sf.parse_json(sf.lit('{}'))).show()
+---+---+-----+
|pos|key|value|
+---+---+-----+
+---+---+-----+