Microsoft.Spark Namespace

Reference

Important

Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

Classes

Broadcast<T>	A broadcast variable. Broadcast variables allow the programmer to keep a read-only variable cached on each machine rather than shipping a copy of it with tasks. They can be used, for example, to give every node a copy of a large input dataset in an efficient manner. Spark also attempts to distribute broadcast variables using efficient broadcast algorithms to reduce communication cost.
DeprecatedAttribute	Custom attribute to denote the Spark version in which an API is deprecated.
JvmException	Contains the message returned from the Microsoft.Spark.Interop.Ipc.JvmBridge on an error.
RemovedAttribute	Custom attribute to denote the Spark version in which an API is removed.
SinceAttribute	Custom attribute to denote the Spark version in which an API is introduced.
SparkConf	Configuration for a Spark application. Used to set various Spark parameters as key-value pairs.
SparkContext	Main entry point for Spark functionality. A SparkContext represents the connection to a Spark cluster, and can be used to create RDDs, accumulators and broadcast variables on that cluster. Only one `SparkContext` should be active per JVM. You must `stop()` the active `SparkContext` before creating a new one.
SparkFiles	Resolves paths to files added through AddFile(String, Boolean).
VersionAttribute	Base class for custom attributes that involve the Spark version.