DataFrameWriter Class

Definition

Namespace:

Microsoft.Spark.Sql

Assembly:

Microsoft.Spark.dll

Package:

Microsoft.Spark v1.0.0

Interface used to write a DataFrame to external storage systems (e.g. file systems, key-value stores, etc).

public sealed class DataFrameWriter

type DataFrameWriter = class

Public NotInheritable Class DataFrameWriter

Inheritance

Object

DataFrameWriter

Methods

BucketBy(Int32, String, String[])	Buckets the output by the given columns. If specified, the output is laid out on the file system similar to Hive's bucketing scheme.
Csv(String)	Saves the content of the DataFrame in CSV format at the specified path.
Format(String)	Specifies the underlying output data source. Built-in options include "parquet", "json", etc.
InsertInto(String)	Inserts the content of the DataFrame to the specified table. It requires that the schema of the DataFrame is the same as the schema of the table.
Jdbc(String, String, Dictionary<String,String>)	Saves the content of the DataFrame to a external database table via JDBC
Json(String)	Saves the content of the DataFrame in JSON format at the specified path.
Mode(SaveMode)	Specifies the behavior when data or table already exists.
Mode(String)	Specifies the behavior when data or table already exists.
Option(String, Boolean)	Adds an output option for the underlying data source.
Option(String, Double)	Adds an output option for the underlying data source.
Option(String, Int64)	Adds an output option for the underlying data source.
Option(String, String)	Adds an output option for the underlying data source.
Options(Dictionary<String,String>)	Adds output options for the underlying data source.
Orc(String)	Saves the content of the DataFrame in ORC format at the specified path.
Parquet(String)	Saves the content of the DataFrame in Parquet format at the specified path.
PartitionBy(String[])	Partitions the output by the given columns on the file system. If specified, the output is laid out on the file system similar to Hive's partitioning scheme.
Save()	Saves the content of the DataFrame as the specified table.
Save(String)	Saves the content of the DataFrame at the specified path.
SaveAsTable(String)	Saves the content of the DataFrame as the specified table.
SortBy(String, String[])	Sorts the output in each bucket by the given columns.
Text(String)	Saves the content of the DataFrame in a text file at the specified path.