What is the purpose of Optimize tab in dataflow transformations other than sink?

Kobi Averbuch 26 Reputation points Microsoft Employee

The Optimize tab exists in many transformations types. I understand the purpose in sink as we want to partitions the results, but why does it do in other intermediate transformations?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,445 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. MarkKromer-MSFT 5,186 Reputation points Microsoft Employee

    At any point inside your data flow, you can tell ADF to repartition data during each transformation step. ADF will honor that setting and repart data at each step. The best scenario for doing this is when you sort. If you wish to maintain that sorting throughout your data flow, set the Sort transformation to single partition.