DataFrame.LocalCheckpoint(Boolean) Method

Reference

Definition

Namespace:: Microsoft.Spark.Sql

Assembly:: Microsoft.Spark.dll

Package:: Microsoft.Spark v1.0.0

Important

Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

Returns a locally checkpointed version of this DataFrame.

public Microsoft.Spark.Sql.DataFrame LocalCheckpoint (bool eager = true);

member this.LocalCheckpoint : bool -> Microsoft.Spark.Sql.DataFrame

Public Function LocalCheckpoint (Optional eager As Boolean = true) As DataFrame

Parameters

eager: Boolean

Whether to checkpoint this DataFrame immediately

Returns

DataFrame

DataFrame object

Remarks

Checkpointing can be used to truncate the logical plan of this `DataFrame`, which is especially useful in iterative algorithms where the plan may grow exponentially. Local checkpoints are written to executor storage and despite potentially faster they are unreliable and may compromise job completion.

Applies to