RelationalGroupedDataset.Apply Method

Definition

Namespace:

Microsoft.Spark.Sql

Assembly:

Microsoft.Spark.dll

Package:

Microsoft.Spark v1.0.0

Overloads

Apply(StructType, Func<RecordBatch,RecordBatch>)

Maps each group of the current DataFrame using a UDF and returns the result as a DataFrame. The user-defined function should take an Apache Arrow RecordBatch and return another Apache Arrow RecordBatch. For each group, all columns are passed together as a RecordBatch to the user-function and the returned RecordBatch are combined as a DataFrame. The returned Apache.Arrow.RecordBatch can be of arbitrary length and its schema must match returnType.

Apply(StructType, Func<DataFrame,DataFrame>)

Maps each group of the current DataFrame using a UDF and returns the result as a DataFrame. The user-defined function should take an DataFrame and return another DataFrame. For each group, all columns are passed together as an DataFrame to the user-function and the returned FxDataFrame are combined as a DataFrame. The returned DataFrame can be of arbitrary length and its schema must match returnType.

Apply(StructType, Func<RecordBatch,RecordBatch>)

Maps each group of the current DataFrame using a UDF and returns the result as a DataFrame.

The user-defined function should take an Apache Arrow RecordBatch and return another Apache Arrow RecordBatch. For each group, all columns are passed together as a RecordBatch to the user-function and the returned RecordBatch are combined as a DataFrame.

The returned Apache.Arrow.RecordBatch can be of arbitrary length and its schema must match returnType.

public Microsoft.Spark.Sql.DataFrame Apply (Microsoft.Spark.Sql.Types.StructType returnType, Func<Apache.Arrow.RecordBatch,Apache.Arrow.RecordBatch> func);

member this.Apply : Microsoft.Spark.Sql.Types.StructType * Func<Apache.Arrow.RecordBatch, Apache.Arrow.RecordBatch> -> Microsoft.Spark.Sql.DataFrame

Public Function Apply (returnType As StructType, func As Func(Of RecordBatch, RecordBatch)) As DataFrame

Parameters

returnType

StructType

The StructType that represents the shape of the return data set.

func

Func<Apache.Arrow.RecordBatch,Apache.Arrow.RecordBatch>

A grouped map user-defined function.

Returns

DataFrame

New DataFrame object with the UDF applied.

Apply(StructType, Func<DataFrame,DataFrame>)

Maps each group of the current DataFrame using a UDF and returns the result as a DataFrame.

The user-defined function should take an DataFrame and return another DataFrame. For each group, all columns are passed together as an DataFrame to the user-function and the returned FxDataFrame are combined as a DataFrame.

The returned DataFrame can be of arbitrary length and its schema must match returnType.

public Microsoft.Spark.Sql.DataFrame Apply (Microsoft.Spark.Sql.Types.StructType returnType, Func<Microsoft.Data.Analysis.DataFrame,Microsoft.Data.Analysis.DataFrame> func);

member this.Apply : Microsoft.Spark.Sql.Types.StructType * Func<Microsoft.Data.Analysis.DataFrame, Microsoft.Data.Analysis.DataFrame> -> Microsoft.Spark.Sql.DataFrame

Public Function Apply (returnType As StructType, func As Func(Of DataFrame, DataFrame)) As DataFrame

Parameters

returnType

StructType

The StructType that represents the schema of the return data set.

func

Func<DataFrame,DataFrame>

A grouped map user-defined function.

Returns

DataFrame

New DataFrame object with the UDF applied.