Kopīgot, izmantojot


window_time

Computes the event time from a window column. The column window values are produced by window aggregating operators and are of type STRUCT<start: TIMESTAMP, end: TIMESTAMP> where start is inclusive and end is exclusive. The event time of records produced by window aggregating operators can be computed as window_time(window) and are window.end - lit(1).alias("microsecond") (as microsecond is the minimal supported event time precision). The window column must be one produced by a window aggregating operator.

For the corresponding Databricks SQL function, see window_time function.

Syntax

from pyspark.databricks.sql import functions as dbf

dbf.window_time(windowColumn=<windowColumn>)

Parameters

Parameter Type Description
windowColumn pyspark.sql.Column or str The window column of a window aggregate records.

Returns

pyspark.sql.Column: the column for computed results.

Examples

import datetime
from pyspark.databricks.sql import functions as dbf
df = spark.createDataFrame([(datetime.datetime(2016, 3, 11, 9, 0, 7), 1)], ['dt', 'v'])
df2 = df.groupBy(dbf.window('dt', '5 seconds')).agg(dbf.sum('v'))