variant_get

Extracts a sub-variant from v according to path, and then cast the sub-variant to targetType. Returns null if the path does not exist. Throws an exception if the cast fails.

Syntax

from pyspark.sql import functions as sf

sf.variant_get(v, path, targetType)

Parameters

Parameter	Type	Description
v	pyspark.sql.Column or str	A variant column or column name.
path	pyspark.sql.Column or str	A column containing the extraction path strings or a string representing the extraction path. A valid path should start with $ and is followed by zero or more segments like [123], .name, ['name'], or ["name"].
targetType	str	The target data type to cast into, in a DDL-formatted string.

Returns

pyspark.sql.Column: a column of targetType representing the extracted result

Examples

from pyspark.sql import functions as sf
df = spark.createDataFrame([ {'json': '''{ "a" : 1 }''', 'path': '$.a'} ])
v = sf.parse_json(df.json)
df.select(sf.variant_get(v, "$.a", "int").alias("r")).collect()

[Row(r=1)]

df.select(sf.variant_get(v, "$.b", "int").alias("r")).collect()

[Row(r=None)]

df.select(sf.variant_get(sf.parse_json(df.json), df.path, "int").alias("r")).collect()

[Row(r=1)]