DataFrame.WithWatermark(String, String) Method

Definition

Namespace:

Microsoft.Spark.Sql

Assembly:

Microsoft.Spark.dll

Package:

Microsoft.Spark v1.0.0

Defines an event time watermark for this DataFrame. A watermark tracks a point in time before which we assume no more late data is going to arrive.

public Microsoft.Spark.Sql.DataFrame WithWatermark (string eventTime, string delayThreshold);

member this.WithWatermark : string * string -> Microsoft.Spark.Sql.DataFrame

Public Function WithWatermark (eventTime As String, delayThreshold As String) As DataFrame

Parameters

eventTime

String

The name of the column that contains the event time of the row.

delayThreshold

String

The minimum delay to wait to data to arrive late, relative to the latest record that has been processed in the form of an interval (e.g. "1 minute" or "5 hours").

Returns

DataFrame

DataFrame object