Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Computes the event time from a window column. The column window values are produced by window aggregating operators and are of type STRUCT<start: TIMESTAMP, end: TIMESTAMP> where start is inclusive and end is exclusive. The event time of records produced by window aggregating operators can be computed as window_time(window) and are window.end - lit(1).alias("microsecond") (as microsecond is the minimal supported event time precision). The window column must be one produced by a window aggregating operator.
For the corresponding Databricks SQL function, see window_time function.
Syntax
from pyspark.databricks.sql import functions as dbf
dbf.window_time(windowColumn=<windowColumn>)
Parameters
| Parameter | Type | Description |
|---|---|---|
windowColumn |
pyspark.sql.Column or str |
The window column of a window aggregate records. |
Returns
pyspark.sql.Column: the column for computed results.
Examples
import datetime
from pyspark.databricks.sql import functions as dbf
df = spark.createDataFrame([(datetime.datetime(2016, 3, 11, 9, 0, 7), 1)], ['dt', 'v'])
df2 = df.groupBy(dbf.window('dt', '5 seconds')).agg(dbf.sum('v'))