DataFrame.Checkpoint(Boolean) Method
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Returns a checkpointed version of this DataFrame.
public Microsoft.Spark.Sql.DataFrame Checkpoint (bool eager = true);member this.Checkpoint : bool -> Microsoft.Spark.Sql.DataFramePublic Function Checkpoint (Optional eager As Boolean = true) As DataFrameParameters
- eager
- Boolean
Whether to checkpoint this DataFrame immediately
Returns
Checkpointed DataFrame
Remarks
Checkpointing can be used to truncate the logical plan of this `DataFrame`, which is especially useful in iterative algorithms where the plan may grow exponentially. It will be saved to files inside the checkpoint directory set with SetCheckpointDir(String).