get (Observation)

Gets the observed metrics.

Waits until the observed dataset finishes its first action. Only the result of the first action is available. Subsequent actions do not modify the result.

Syntax

get

Returns

dict

Examples

from pyspark.sql.functions import col, count, lit, max
from pyspark.sql import Observation

df = spark.createDataFrame([["Alice", 2], ["Bob", 5]], ["name", "age"])
observation = Observation("my metrics")
observed_df = df.observe(observation, count(lit(1)).alias("count"), max(col("age")))
observed_df.count()
# 2
observation.get
# {'count': 2, 'max(age)': 5}