Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Sets the trigger for the streaming query. If not set, the query runs as fast as possible, equivalent to processingTime='0 seconds'. Only one trigger parameter can be set at a time.
See Configure Structured Streaming trigger intervals.
Compute and pipeline execution
In Lakeflow pipelines, the pipeline manages stream execution. Use pipeline execution settings to control when pipeline data is processed.
When you use DataStreamWriter directly on serverless compute for notebooks and jobs, use availableNow=True. The default trigger, processingTime, continuous, and realTime are not supported on that compute. These restrictions do not apply to serverless pipeline execution modes. See Streaming limitations.
Syntax
trigger(*, processingTime=None, once=None, continuous=None, availableNow=None, realTime=None)
Parameters
| Parameter | Type | Description |
|---|---|---|
processingTime |
str, optional | A processing time interval string (for example, '5 seconds', '1 minute'). Runs a microbatch query periodically based on the processing time. |
once |
bool, optional | If True, processes only one batch of data then terminates the query. Deprecated in Databricks Runtime 11.3 LTS and above. Use availableNow=True instead. |
continuous |
str, optional | Not supported on Azure Databricks. This parameter enables Apache Spark's experimental Continuous Processing mode, which differs from continuous pipeline mode. Use real-time mode for low-latency Structured Streaming queries. |
availableNow |
bool, optional | If True, processes all available data in multiple batches then terminates the query. |
realTime |
str, optional | A batch duration string (for example, '5 minutes'). Processes records as they arrive within a long-running batch. The value controls batch duration, not record-processing latency. See Real-time mode in Structured Streaming. |
Returns
DataStreamWriter
Examples
df = spark.readStream.format("rate").load()
Trigger execution every 5 seconds:
df.writeStream.trigger(processingTime='5 seconds')
# <...streaming.readwriter.DataStreamWriter object ...>
Process all available data in multiple batches:
df.writeStream.trigger(availableNow=True)
# <...streaming.readwriter.DataStreamWriter object ...>
Configure real-time mode with five-minute batches. Records are processed as they arrive during each batch. Use a supported source, sink, and compute type:
kafka_df = (spark.readStream
.format("kafka")
.option("kafka.bootstrap.servers", "<bootstrap-servers>")
.option("subscribe", "<input-topic>")
.load())
kafka_df.writeStream.trigger(realTime="5 minutes")
# <...streaming.readwriter.DataStreamWriter object ...>