Bemærk
Adgang til denne side kræver godkendelse. Du kan prøve at logge på eller ændre mapper.
Adgang til denne side kræver godkendelse. Du kan prøve at ændre mapper.
Manages all active StreamingQuery instances associated with a SparkSession. Use spark.streams to access this.
Syntax
# Access through SparkSession
spark.streams
Properties
| Property | Description |
|---|---|
active |
Returns a list of all active streaming queries associated with this SparkSession. |
Methods
| Method | Description |
|---|---|
get(id) |
Returns an active query by its unique ID. |
awaitAnyTermination(timeout) |
Waits until any active query terminates, or until the timeout expires. |
resetTerminated() |
Forgets past terminated queries so that awaitAnyTermination() can be used again to wait for new terminations. |
addListener(listener) |
Registers a StreamingQueryListener to receive lifecycle event callbacks. |
removeListener(listener) |
Deregisters a StreamingQueryListener. |
Examples
sdf = spark.readStream.format("rate").load()
sq = sdf.writeStream.format('memory').queryName('this_query').start()
sqm = spark.streams
[q.name for q in sqm.active]
# ['this_query']
sqm.awaitAnyTermination(5)
# True
sq.stop()
sqm.resetTerminated()