Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Extracts a quantile value from a KLL float sketch given an input rank value. The rank can be a single value or an array.
Syntax
from pyspark.sql import functions as sf
sf.kll_sketch_get_quantile_float(sketch, rank)
Parameters
| Parameter | Type | Description |
|---|---|---|
sketch |
pyspark.sql.Column or str |
The KLL float sketch binary representation. |
rank |
pyspark.sql.Column or str |
The rank value(s) to extract (between 0.0 and 1.0). |
Returns
pyspark.sql.Column: The quantile value(s).
Examples
Example 1: Get quantile from KLL float sketch
from pyspark.sql import functions as sf
df = spark.createDataFrame([1.0,2.0,3.0,4.0,5.0], "FLOAT")
sketch_df = df.agg(sf.kll_sketch_agg_float("value").alias("sketch"))
sketch_df.select(sf.kll_sketch_get_quantile_float("sketch", sf.lit(0.5))).show()
+------------------------------------------+
|kll_sketch_get_quantile_float(sketch, 0.5)|
+------------------------------------------+
| 3.0|
+------------------------------------------+