Edit

Share via

Array functions in mapping data flows

APPLIES TO: Azure Data Factory Azure Synapse Analytics

Tip

Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!

Data flows are available in both Azure Data Factory pipelines and Azure Synapse Analytics pipelines. This article applies to mapping data flows. If you're new to transformations, refer to the introductory article Transform data using mapping data flows.

This article provides details about array functions supported by Azure Data Factory and Azure Synapse Analytics in mapping data flows.

Array function list

Array functions perform transformations on data structures that are arrays. Special keywords address array elements and indexes:

  • #acc represents a value that you want to include in your single output when you reduce an array.
  • #index represents the current array index, along with array index numbers #index2, #index3 ....
  • #item represents the current element value in the array.
Array function Task
array Creates an array of items. All items should be of the same type. If no items are specified, an empty string array is the default. Same as a [] creation operator.
at Finds the element at an array index. The index is 1 based. Out-of-bounds index results in a null value. Finds a value in a map when given a key. If the key isn't found, it returns null.
contains Returns true if any element in the provided array evaluates as true in the provided predicate. The contains function expects a reference to one element in the predicate function as #item.
distinct Returns a distinct set of items from an array.
except Returns a difference set of one array from another dropping duplicates.
filter Filters elements out of the array that don't meet the provided predicate. The filter function expects a reference to one element in the predicate function as #item.
find Finds the first item from an array that matches the condition. It takes a filter function where you can address the item in the array as #item. For deeply nested maps, you can refer to the parent maps by using the #item_n(#item_1, #item_2...) notation.
flatten Flattens an array or arrays into a single array. Arrays of atomic items are returned unaltered. The last argument is optional and is defaulted to false to flatten recursively more than one level deep.
in Checks if an item is in the array.
intersect Returns an intersection set of distinct items from two arrays.
map Maps each element of the array to a new element by using the provided expression. The map function expects a reference to one element in the expression function as #item.
mapIf Conditionally maps an array to another array of the same or smaller length. The values can be of any data type, including structTypes. It takes a mapping function where you can address the item in the array as #item and the current index as #index. For deeply nested maps, you can refer to the parent maps by using the #item_[n](#item_1, #index_1...) notation.
mapIndex Maps each element of the array to a new element by using the provided expression. The map function expects a reference to one element in the expression function as #item and a reference to the element index as #index.
mapLoop Loops through from 1 to length to create an array of that length. It takes a mapping function where you can address the index in the array as #index. For deeply nested maps, you can refer to the parent maps by using the #index_n(#index_1, #index_2...) notation.
reduce Accumulates elements in an array. The reduce function expects a reference to an accumulator and one element in the first expression function as #acc and #item. It expects the resulting value as #result to be used in the second expression function.
size Finds the size of an array or map type.
slice Extracts a subset of an array from a position. Position is 1 based. If the length is omitted, it defaults to the end of the string.
sort Sorts the array by using the provided predicate function. The sort function expects a reference to two consecutive elements in the expression function as #item1 and #item2.
unfold Unfolds an array into a set of rows and repeats the values for the remaining columns in every row.
union Returns a union set of distinct items from two arrays.

Other available functions

Related content

  • Last updated on