DataFrame Class

Definition

A DataFrame to support indexing, binary operations, sorting, selection and other APIs. This will eventually also expose an IDataView for ML.NET

public class DataFrame : Microsoft.ML.IDataView
type DataFrame = class
    interface IDataView
Public Class DataFrame
Implements IDataView
Inheritance
DataFrame
Implements

Constructors

DataFrame(DataFrameColumn[])
DataFrame(IEnumerable<DataFrameColumn>)

Constructs a DataFrame with columns.

Properties

Columns

Returns the columns contained in the DataFrame as a DataFrameColumnCollection

Item[IEnumerable<Boolean>]

Returns a new DataFrame using the boolean values in rowFilter

Item[IEnumerable<Int32>]

Returns a new DataFrame using the row indices in rowIndices

Item[IEnumerable<Int64>]

Returns a new DataFrame using the row indices in rowIndices

Item[Int64, Int32]

An Indexer to get or set values.

Item[PrimitiveDataFrameColumn<Boolean>]

Returns a new DataFrame using the boolean values in filter

Item[PrimitiveDataFrameColumn<Int32>]

Returns a new DataFrame using the row indices in rowIndices

Item[PrimitiveDataFrameColumn<Int64>]

Returns a new DataFrame using the row indices in rowIndices

Item[String]

An indexer based on Name

Rows

Returns a DataFrameRowCollection that contains a view of the rows in this DataFrame

Methods

Add<T>(IReadOnlyList<T>, Boolean)
Add<T>(T, Boolean)

Performs an element-wise addition on each column

AddPrefix(String, Boolean)

Adds a prefix to the column names

AddSuffix(String, Boolean)

Adds a suffix to the column names

And(Boolean, Boolean)

Performs an element-wise boolean And on each column

And(IReadOnlyList<Boolean>, Boolean)
Append(IEnumerable<DataFrameRow>, Boolean, CultureInfo)

Appends rows to the DataFrame

Append(IEnumerable<KeyValuePair<String,Object>>, Boolean, CultureInfo)

Appends a row by enumerating column names and values from row

Append(IEnumerable<Object>, Boolean, CultureInfo)

Appends a row to the DataFrame

Clamp<U>(U, U, Boolean)

Clamps values beyond the specified thresholds on numeric columns

Clone()

Returns a full copy

Description()

Generates descriptive statistics that summarize each numeric column

Divide<T>(IReadOnlyList<T>, Boolean)
Divide<T>(T, Boolean)

Performs an element-wise division on each column

DropNulls(DropNullOptions)

Returns a DataFrame with no missing values

ElementwiseEquals<T>(IReadOnlyList<T>)
ElementwiseEquals<T>(T)

Performs an element-wise equals on each column

ElementwiseGreaterThan<T>(IReadOnlyList<T>)
ElementwiseGreaterThan<T>(T)

Performs an element-wise greater than on each column

ElementwiseGreaterThanOrEqual<T>(IReadOnlyList<T>)
ElementwiseGreaterThanOrEqual<T>(T)

Performs an element-wise greater than or equal on each column

ElementwiseLessThan<T>(IReadOnlyList<T>)
ElementwiseLessThan<T>(T)

Performs an element-wise less than on each column

ElementwiseLessThanOrEqual<T>(IReadOnlyList<T>)
ElementwiseLessThanOrEqual<T>(T)

Performs an element-wise less than or equal on each column

ElementwiseNotEquals<T>(IReadOnlyList<T>)
ElementwiseNotEquals<T>(T)

Performs an element-wise not-equals on each column

FillNulls(IList<Object>, Boolean)

Fills null values in each column with values from values.

FillNulls(Object, Boolean)

Fills null values with value.

Filter(PrimitiveDataFrameColumn<Boolean>)

Returns a new DataFrame using the boolean values in filter

Filter(PrimitiveDataFrameColumn<Int32>)

Returns a new DataFrame using the row indices in rowIndices

Filter(PrimitiveDataFrameColumn<Int64>)

Returns a new DataFrame using the row indices in rowIndices

FromArrowRecordBatch(RecordBatch)

Wraps a DataFrame around an Arrow Apache.Arrow.RecordBatch without copying data

FromSchema(DbDataReader)
GroupBy(String)

Groups the rows of the DataFrame by unique values in the columnName column.

GroupBy<TKey>(String)

Groups the rows of the DataFrame by unique values in the columnName column.

Head(Int32)

Returns the first numberOfRows rows

Info()

Generates a concise summary of each column in the DataFrame

Join(DataFrame, String, String, JoinAlgorithm)

Joins columns of another DataFrame

LeftShift(Int32, Boolean)

Performs an element-wise left shift on each column

LoadCsv(Stream, Char, Boolean, String[], Type[], Int64, Int32, Boolean, Encoding, Boolean, CultureInfo)

Reads a seekable stream of CSV data into a DataFrame.

LoadCsv(String, Char, Boolean, String[], Type[], Int32, Int32, Boolean, Encoding, Boolean, CultureInfo)

Reads a text file as a DataFrame.

LoadCsvFromString(String, Char, Boolean, String[], Type[], Int64, Int32, Boolean, Boolean, CultureInfo)

Reads CSV data passed in as a string into a DataFrame.

LoadFrom(DbDataAdapter)
LoadFrom(DbDataReader)
LoadFrom(IEnumerable<IList<Object>>, IList<ValueTuple<String,Type>>)
Merge(DataFrame, String[], String[], String, String, JoinAlgorithm)
Merge<TKey>(DataFrame, String, String, String, String, JoinAlgorithm)

Merge DataFrames with a database style join (for backward compatibility)

Modulo<T>(IReadOnlyList<T>, Boolean)
Modulo<T>(T, Boolean)

Performs an element-wise modulus operation on each column

Multiply<T>(IReadOnlyList<T>, Boolean)
Multiply<T>(T, Boolean)

Performs an element-wise multiplication on each column

Or(Boolean, Boolean)

Performs an element-wise boolean Or on each column

Or(IReadOnlyList<Boolean>, Boolean)
OrderBy(String)

Orders the data frame by a specified column.

OrderByDescending(String)

Orders the data frame by a specified column in descending order.

ReverseAdd<T>(T, Boolean)

Performs a reversed element-wise addition on each column

ReverseAnd(Boolean, Boolean)

Performs a reversed element-wise boolean And on each column

ReverseDivide<T>(T, Boolean)

Performs a reversed element-wise division on each column

ReverseModulo<T>(T, Boolean)

Performs a reversed element-wise modulus operation on each column

ReverseMultiply<T>(T, Boolean)

Performs a reversed element-wise multiplication on each column

ReverseOr(Boolean, Boolean)

Performs a reversed element-wise boolean Or on each column

ReverseSubtract<T>(T, Boolean)

Performs a reversed element-wise subtraction on each column

ReverseXor(Boolean, Boolean)

Performs a reversed element-wise boolean Xor on each column

RightShift(Int32, Boolean)

Performs an element-wise right shift on each column

Sample(Int32)

Returns a random sample of rows

SaveCsv(DataFrame, Stream, Char, Boolean, Encoding, CultureInfo)

Saves a DataFrame into a CSV.

SaveCsv(DataFrame, String, Char, Boolean, Encoding, CultureInfo)

Saves a DataFrame into a CSV.

SaveTo(DataTable)
SaveTo(DbDataAdapter, DbProviderFactory)
Subtract<T>(IReadOnlyList<T>, Boolean)
Subtract<T>(T, Boolean)

Performs an element-wise subtraction on each column

Tail(Int32)

Returns the last numberOfRows rows

ToArrowRecordBatches()

Returns an IEnumerable<T> mostly without copying data

ToString()

A preview of the contents of this DataFrame as a string.

ToTable()
WriteCsv(DataFrame, Stream, Char, Boolean, Encoding, CultureInfo)
Obsolete.

Writes a DataFrame into a CSV.

WriteCsv(DataFrame, String, Char, Boolean, Encoding, CultureInfo)
Obsolete.

Writes a DataFrame into a CSV.

Xor(Boolean, Boolean)

Performs an element-wise boolean Xor on each column

Xor(IReadOnlyList<Boolean>, Boolean)

Operators

Addition(Byte, DataFrame)
Addition(DataFrame, Byte)
Addition(DataFrame, Decimal)
Addition(DataFrame, Double)
Addition(DataFrame, Int16)
Addition(DataFrame, Int32)
Addition(DataFrame, Int64)
Addition(DataFrame, SByte)
Addition(DataFrame, Single)
Addition(DataFrame, UInt16)
Addition(DataFrame, UInt32)
Addition(DataFrame, UInt64)
Addition(Decimal, DataFrame)
Addition(Double, DataFrame)
Addition(Int16, DataFrame)
Addition(Int32, DataFrame)
Addition(Int64, DataFrame)
Addition(SByte, DataFrame)
Addition(Single, DataFrame)
Addition(UInt16, DataFrame)
Addition(UInt32, DataFrame)
Addition(UInt64, DataFrame)
Division(Byte, DataFrame)
Division(DataFrame, Byte)
Division(DataFrame, Decimal)
Division(DataFrame, Double)
Division(DataFrame, Int16)
Division(DataFrame, Int32)
Division(DataFrame, Int64)
Division(DataFrame, SByte)
Division(DataFrame, Single)
Division(DataFrame, UInt16)
Division(DataFrame, UInt32)
Division(DataFrame, UInt64)
Division(Decimal, DataFrame)
Division(Double, DataFrame)
Division(Int16, DataFrame)
Division(Int32, DataFrame)
Division(Int64, DataFrame)
Division(SByte, DataFrame)
Division(Single, DataFrame)
Division(UInt16, DataFrame)
Division(UInt32, DataFrame)
Division(UInt64, DataFrame)
LeftShift(DataFrame, Int32)
Modulus(Byte, DataFrame)
Modulus(DataFrame, Byte)
Modulus(DataFrame, Decimal)
Modulus(DataFrame, Double)
Modulus(DataFrame, Int16)
Modulus(DataFrame, Int32)
Modulus(DataFrame, Int64)
Modulus(DataFrame, SByte)
Modulus(DataFrame, Single)
Modulus(DataFrame, UInt16)
Modulus(DataFrame, UInt32)
Modulus(DataFrame, UInt64)
Modulus(Decimal, DataFrame)
Modulus(Double, DataFrame)
Modulus(Int16, DataFrame)
Modulus(Int32, DataFrame)
Modulus(Int64, DataFrame)
Modulus(SByte, DataFrame)
Modulus(Single, DataFrame)
Modulus(UInt16, DataFrame)
Modulus(UInt32, DataFrame)
Modulus(UInt64, DataFrame)
Multiply(Byte, DataFrame)
Multiply(DataFrame, Byte)
Multiply(DataFrame, Decimal)
Multiply(DataFrame, Double)
Multiply(DataFrame, Int16)
Multiply(DataFrame, Int32)
Multiply(DataFrame, Int64)
Multiply(DataFrame, SByte)
Multiply(DataFrame, Single)
Multiply(DataFrame, UInt16)
Multiply(DataFrame, UInt32)
Multiply(DataFrame, UInt64)
Multiply(Decimal, DataFrame)
Multiply(Double, DataFrame)
Multiply(Int16, DataFrame)
Multiply(Int32, DataFrame)
Multiply(Int64, DataFrame)
Multiply(SByte, DataFrame)
Multiply(Single, DataFrame)
Multiply(UInt16, DataFrame)
Multiply(UInt32, DataFrame)
Multiply(UInt64, DataFrame)
RightShift(DataFrame, Int32)
Subtraction(Byte, DataFrame)
Subtraction(DataFrame, Byte)
Subtraction(DataFrame, Decimal)
Subtraction(DataFrame, Double)
Subtraction(DataFrame, Int16)
Subtraction(DataFrame, Int32)
Subtraction(DataFrame, Int64)
Subtraction(DataFrame, SByte)
Subtraction(DataFrame, Single)
Subtraction(DataFrame, UInt16)
Subtraction(DataFrame, UInt32)
Subtraction(DataFrame, UInt64)
Subtraction(Decimal, DataFrame)
Subtraction(Double, DataFrame)
Subtraction(Int16, DataFrame)
Subtraction(Int32, DataFrame)
Subtraction(Int64, DataFrame)
Subtraction(SByte, DataFrame)
Subtraction(Single, DataFrame)
Subtraction(UInt16, DataFrame)
Subtraction(UInt32, DataFrame)
Subtraction(UInt64, DataFrame)

Explicit Interface Implementations

IDataView.CanShuffle
IDataView.GetRowCount()
IDataView.GetRowCursor(IEnumerable<DataViewSchema.Column>, Random)
IDataView.GetRowCursorSet(IEnumerable<DataViewSchema.Column>, Int32, Random)
IDataView.Schema

Extension Methods

ToDataFrame(IDataView, Int64)

Returns a DataFrame from this dataView.

ToDataFrame(IDataView, Int64, String[])

Returns a DataFrame with the first maxRows of this dataView.

ToDataFrame(IDataView, String[])

Returns a DataFrame with the first 100 rows of this dataView.

Applies to