How to sort an array variable in ADF

Question

How to sort an array variable in ADF

Priya Jha 896

HI All,

I have an array variable with value [5,8,3,10,1,5] i need to sort values of this array in ascending order like this [1,3,5,5,8,10]

Any way to achieve this via ADF pipeline or dataflows?

Accepted answer

2 additional answers

Your answer

Answer 1

Hello @Priya Jha ,
Thanks for the question and using MS Q&A platform.

Sorting in a pipeline:
There is no function to do so at this time. Implementing some sorting algorithm using just pipeline activities is not practical. It would also be expensive. Generally it is best to do the sort outside of Data Factory in that case.

Sorting in a Dataflow:
There is a sort function in Dataflow. You could even pass the variable into the dataflow and then operate on that to make new dataflow parameter. However, remember Dataflow is kinda like Copy data activity in that it expects a source and sink. If all you want to do is sort the variable and not move data, it isn't really good use of resources. Also I don't think there is an output for dataflow parameters back into the pipeline. You could write the array to a file, and read that into dataflow and then write it back out once sorted.

Please do let me if you have any queries.

Thanks
Martin

Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
- If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Answer 2

Maatoug wassim 76

Hi, there is a similar answered question (azure-data-factory-sort-values-in-the-array-variab.html)
Hope it helps.

MartinJaffer-MSFT 26,236 Reputation points

2022-12-09T18:08:41.06+00:00

Thanks for the assist @Maatoug wassim , keep up the good work!

Answer 3

Nandan Hegde 36,151 MVP Volunteer Moderator

Hey

@MartinJaffer-MSFT

The below blog helps to sort an array via regular expressions :

https://datasharkx.wordpress.com/2023/02/25/sort-an-array-in-azure-data-factory-synapse-pipeline-via-regular-expressions/

Can you provide your thoughts on that? Or am I missing out anything

MartinJaffer-MSFT 26,236 Reputation points

2023-03-07T19:13:38.1666667+00:00

@Nandan Hegde Thank you for creating this how-to article. I have noted a couple issues.

Firstly, the title may confuse some people. Regular Expressions (RegEx) is a known thing for matching patterns in text, more complicated than finding substrings. I think you meant to use pipeline expressions?

Secondly, in your code snippet, j = -1 and j += 1 appear to be in the same block of code, they happen one after another. I think you meant for the j = -1 to be part of the if (InputAr ...1): predicate, and the j += 1 to be in the outer while loop. You can indent each line of code to help clarify where the if block ends, and where the while block ends. As it is, everything appears to be on the same level.

Using an array instead of an integer counter variable (in Iteration) is novel and I had to think about it for a bit. Very cool!

Share via

How to sort an array variable in ADF

2 additional answers

Your answer