Share via

Azure DataFactory Dataflow Activity read

Nirmal Chowdhary 0 Reputation points
2024-05-23T16:11:45.6933333+00:00

Hello, I have a question regarding the use of the Data Flow activity in Azure Data Factory (ADF). I'm looking to pass data to the Data Flow activity through a foreach iteration. My scenario involves performing a join between two datasets, filtering some data, and then passing the result to another activity. However, this operation needs to run multiple times within a foreach loop.

The challenge I'm facing is that the two source datasets used in this operation are static, and I don't want to load them each time the foreach loop runs. Is there a way to pass this data to the foreach loop and reuse it multiple times within the Data Flow activity?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,893 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. ShaikMaheer-MSFT 38,301 Reputation points Microsoft Employee
    2024-05-24T05:49:11.69+00:00

    Hi Nirmal Chowdhary,

    Thank you for posting query in Microsoft Q&A Platform.

    Consider using parameterized datasets in your Dataflow and inside the loop for each iteration pass the values into parameters accordingly.

    Please check below video to understand about parameterized datasets in ADF.

    Parameterize dataset in ADF

    Hope this helps. Please let me know if any further queries.

    Please consider hitting Accept Answer button. Accepted answers help community as well. Thank you.