How to get the value of a column in Azure Data Factory-Data Flow Activity

Question

Setup: I have a table with 4 columns, table_schema, table_name, column_name and build_sql_query

Issue: In the build_sql_query column I have a sql query built. I need to pass this sql query as a parameter and execute it to get the results. How can I achieve this using data flow mapping?.

Trials: I achieved this using copy activity where I used the lookup activity and passed it parameter into another lookup activity for query option and I was able to get the results. However I want to do this in data flow mapping.

I tried to use the derived column transformation and tried to retrieve build_sql_query values using columns() but it throws me an error right away "The 'columns' function must be enclosed in another function"

Answer 1

You should read this table using a lookup activity and use a foreach to iterate and call the dataflow activity with these as parameters. In the dataflow you can use a source(keep the table/schema & query) parameterized.

Dataflow will be adding inbuilt support for this. Keep tuned for updates.