kll_sketch_agg_float

Aggregate function: returns the compact binary representation of the Datasketches KllFloatsSketch built with the values in the input column. The optional k parameter controls the size and accuracy of the sketch (default 200, range 8-65535).

Syntax

from pyspark.sql import functions as dbf

dbf.kll_sketch_agg_float(col=<col>, k=<k>)

Parameters

Parameter	Type	Description
col	pyspark.sql.Column or column name	The column containing float values to aggregate.
k	pyspark.sql.Column or int, optional	The k parameter that controls size and accuracy (default 200, range 8-65535).

Returns

pyspark.sql.Column: The binary representation of the KllFloatsSketch.

Examples

from pyspark.sql import functions as dbf
df = spark.createDataFrame([1.0,2.0,3.0,4.0,5.0], "FLOAT")
result = df.agg(dbf.kll_sketch_agg_float("value")).first()[0]
result is not None and len(result) > 0

True