schema_merge plugin

Applies to: ✅ Microsoft Fabric ✅ Azure Data Explorer

Merges tabular schema definitions into a unified schema.

Schema definitions are expected to be in the format produced by the getschema operator.

The schema merge operation joins columns in input schemas and tries to reduce data types to common ones. If data types can't be reduced, an error is displayed on the problematic column.

The plugin is invoked with the evaluate operator.

Syntax

T | evaluate schema_merge(PreserveOrder)

Learn more about syntax conventions.

Parameters

Name	Type	Required	Description
PreserveOrder	bool		When set to true, directs the plugin to validate the column order as defined by the first tabular schema that is kept. If the same column is in several schemas, the column ordinal must be like the column ordinal of the first schema that it appeared in. Default value is true.

Returns

The schema_merge plugin returns output similar to what getschema operator returns.

Examples

Merge with a schema that has a new column appended.

Run the query

let schema1 = datatable(Uri:string, HttpStatus:int)[] | getschema;
let schema2 = datatable(Uri:string, HttpStatus:int, Referrer:string)[] | getschema;
union schema1, schema2 | evaluate schema_merge()

Output

ColumnName	ColumnOrdinal	DataType	ColumnType
Uri	0	System.String	string
HttpStatus	1	System.Int32	int
Referrer	2	System.String	string

Merge with a schema that has different column ordering (HttpStatus ordinal changes from 1 to 2 in the new variant).

Run the query

let schema1 = datatable(Uri:string, HttpStatus:int)[] | getschema;
let schema2 = datatable(Uri:string, Referrer:string, HttpStatus:int)[] | getschema;
union schema1, schema2 | evaluate schema_merge()

Output

ColumnName	ColumnOrdinal	DataType	ColumnType
Uri	0	System.String	string
Referrer	1	System.String	string
HttpStatus	-1	ERROR(unknown CSL type:ERROR(columns are out of order))	ERROR(columns are out of order)

Merge with a schema that has different column ordering, but with PreserveOrder set to false.

Run the query

let schema1 = datatable(Uri:string, HttpStatus:int)[] | getschema;
let schema2 = datatable(Uri:string, Referrer:string, HttpStatus:int)[] | getschema;
union schema1, schema2 | evaluate schema_merge(PreserveOrder = false)

Output

ColumnName	ColumnOrdinal	DataType	ColumnType
Uri	0	System.String	string
Referrer	1	System.String	string
HttpStatus	2	System.Int32	int