Share via

Fabric Data Factory - Dynamic Jobs for data ingestion

Ganesan, Jothy 100 Reputation points
2024-06-26T11:04:32.76+00:00

Hi Team,

We have a list of customized SQL queries as source and the output to be loaded into lakehouse as new table.

Is there a way that we could do this with Parameters where we could pass the entire query as a parameter along with the target table name as well as a parameter in Fabric data factory using Data flow and not using notebook? so that we can create only one template and reuse it? inside dataflow we dont have any other transformations.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,977 questions
Microsoft Fabric Training
Microsoft Fabric Training
Microsoft Fabric: A Microsoft unified data platform.Training: Instruction to develop new skills.
21 questions
0 comments
Accepted answer
  1. Nandan Hegde 30,556 Reputation points MVP
    2024-06-26T13:45:25.67+00:00

    Unfortunately , based on my understanding as of today Dataflow Gen 2 cannot be parameterized and that functionality is not available.

    So for you to make it dynamic, you have to use Copy activity in fabric data pipeline wherein your source would be SQL database and you can pass the query as the source and sink as lakehouse below wherein you can make the table name dynamic

    User's image

    0 comments

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest