Piezīmes
Lai piekļūtu šai lapai, ir nepieciešama autorizācija. Varat mēģināt pierakstīties vai mainīt direktorijus.
Lai piekļūtu šai lapai, ir nepieciešama autorizācija. Varat mēģināt mainīt direktorijus.
Manages all active StreamingQuery instances associated with a SparkSession. Use spark.streams to access this.
Syntax
# Access through SparkSession
spark.streams
Properties
| Property | Description |
|---|---|
active |
Returns a list of all active streaming queries associated with this SparkSession. |
Methods
| Method | Description |
|---|---|
get(id) |
Returns an active query by its unique ID. |
awaitAnyTermination(timeout) |
Waits until any active query terminates, or until the timeout expires. |
resetTerminated() |
Forgets past terminated queries so that awaitAnyTermination() can be used again to wait for new terminations. |
addListener(listener) |
Registers a StreamingQueryListener to receive lifecycle event callbacks. |
removeListener(listener) |
Deregisters a StreamingQueryListener. |
Examples
sdf = spark.readStream.format("rate").load()
sq = sdf.writeStream.format('memory').queryName('this_query').start()
sqm = spark.streams
[q.name for q in sqm.active]
# ['this_query']
sqm.awaitAnyTermination(5)
# True
sq.stop()
sqm.resetTerminated()