DataFrameReader Class

Definition

Namespace:

Microsoft.Spark.Sql

Assembly:

Microsoft.Spark.dll

Package:

Microsoft.Spark v1.0.0

DataFrameReader provides functionality to load a DataFrame from external storage systems (e.g. file systems, key-value stores, etc).

public sealed class DataFrameReader

type DataFrameReader = class

Public NotInheritable Class DataFrameReader

Inheritance

Object

DataFrameReader

Methods

Csv(String[])	Loads CSV files and returns the result as a DataFrame.
Format(String)	Specifies the input data source format.
Jdbc(String, String, Dictionary<String,String>)	Construct a DataFrame representing the database table accessible via JDBC URL url named table and connection properties.
Jdbc(String, String, IEnumerable<String>, Dictionary<String,String>)	Construct a DataFrame representing the database table accessible via JDBC URL url named table and connection properties. The predicates parameter gives a list expressions suitable for inclusion in WHERE clauses; each one defines one partition of the DataFrame.
Jdbc(String, String, String, Int64, Int64, Int32, Dictionary<String,String>)	Construct a DataFrame representing the database table accessible via JDBC URL url named table. Partitions of the table will be retrieved in parallel based on the parameters passed to this function.
Json(String[])	Loads a JSON file (one object per line) and returns the result as a DataFrame.
Load()	Loads input in as a DataFrame, for data sources that don't require a path (e.g. external key-value stores).
Load(String)	Loads input in as a DataFrame, for data sources that require a path (e.g. data backed by a local or distributed file system).
Load(String[])	Loads input in as a DataFrame from the given paths.
Option(String, Boolean)	Adds an input option for the underlying data source.
Option(String, Double)	Adds an input option for the underlying data source.
Option(String, Int64)	Adds an input option for the underlying data source.
Option(String, String)	Adds an input option for the underlying data source.
Options(Dictionary<String,String>)	Adds input options for the underlying data source.
Orc(String[])	Loads an ORC file and returns the result as a DataFrame.
Parquet(String[])	Loads a Parquet file, returning the result as a DataFrame.
Schema(String)	Specifies the schema by using the given DDL-formatted string.
Schema(StructType)	Specifies the schema by using StructType.
Table(String)	Returns the specified table as a DataFrame.
Text(String[])	Loads text files and returns a DataFrame whose schema starts with a string column named "value", and followed by partitioned columns if there are any.