Fabric Data Factory - Dynamic Jobs for data ingestion

Ganesan, Jothy 100 Reputation points

Hi Team,

We have a list of customized SQL queries as source and the output to be loaded into lakehouse as new table.

Is there a way that we could do this with Parameters where we could pass the entire query as a parameter along with the target table name as well as a parameter in Fabric data factory using Data flow and not using notebook? so that we can create only one template and reuse it? inside dataflow we dont have any other transformations.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,112 questions
Microsoft Fabric Training
Microsoft Fabric Training
Microsoft Fabric: A Microsoft unified data platform.Training: Instruction to develop new skills.
27 questions
0 comments No comments
{count} votes

Accepted answer
  1. Nandan Hegde 31,346 Reputation points MVP

    Unfortunately , based on my understanding as of today Dataflow Gen 2 cannot be parameterized and that functionality is not available.

    So for you to make it dynamic, you have to use Copy activity in fabric data pipeline wherein your source would be SQL database and you can pass the query as the source and sink as lakehouse below wherein you can make the table name dynamic

    User's image

    0 comments No comments

0 additional answers

Sort by: Most helpful