how to dynamically transform one activity output and feed in another activity in ADF

Question

how to dynamically transform one activity output and feed in another activity in ADF

Arundhati Sen 151

If I directly give (@{item().id} @{item().col1} @{item().col2} @{item().name} @{item().col5}) in set variable is gives back result "C0004\t1000\t\tAT020\t1102"

but I give the activity output (@activity('Lookup_DB_column names').output.firstRow.column_names) which holds nothing but the value (@{item().id} @{item().col1} @{item().col2} @{item().name} @{item().col5}) then it is not giving

"C0004\t1000\t\tAT020\t1102"

but (@{item().id} @{item().col1} @{item().col2} @{item().name} @{item().col5} ) only as string.

Is there any way that activity output or parameter value can be dynamically executed to get result out from the value

Bhargava-MSFT 31,356 Reputation points Microsoft Employee Moderator

2023-03-30T17:28:11.8933333+00:00

Hello Arundhati Sen
Welcome to the MS Q&A platform.

Are you trying to pass the output of an activity as input to another activity in Azure Data Factory (ADF)? And are you trying to pass a string value that you constructed using the expression "@{item().id} @{item().col1} @{item().col2} @{item().name} @{item().col5}" as input to the second activity?

If so, what is the second activity, and what input format does it expect? Can you please provide more details on the activities you are working with and the pipeline flow would be helpful.
Arundhati Sen 151 Reputation points

2023-03-31T05:32:10.5133333+00:00
Sure, my second activity is a web activity (inside a for-each loop) and in it's response body I want to give the expression "@{item().id} @{item().col1} @{item().col2} @{item().name} @{item().col5}"

But for the sake of simplicity, I am taking a set variable activity (in place of web activity) inside For-Each loop and sending the expression "@{item().id} @{item().col1} @{item().col2} @{item().name} @{item().col5}" as a parameter value to test the output.

lookup activity output :

{ "count": 3, "value": [ { "id": "C0004", "col1": "1000", "col2": "", "name": "AT020", "col5": "1102" }, { "id": "C0004", "col1": "1001", "col2": "", "name": "AT020", "col5": "1102" }, { "id": "C0004", "col1": "1100", "col2": "", "name": "AT020", "col5": "1102" } ]

1st set activity inside the For-Each Loop :

provoding parameter dynamic_value which holds the value : @{item().id} @{item().col1} @{item().col2} @{item().name} @{item().col5}

and it gives output :

"value": "@{item().id}\\t@{item().col1}\\t@{item().col2}\\t@{item().name}\\t@{item().col5}"

2nd set activity inside the For-Each Loop :

providing value : @{item().id} @{item().col1} @{item().col2} @{item().name} @{item().col5}

and it gives output :

"value": "C0004\t1100\t\tAT020\t1102"

I want the 1st set activity output same as 2nd set activity output. And to dynamically infer and execute the value given in the parameter and provide output of the value that is given in the parameter.

parameter dynamic_value which holds the value : @{item().id} @{item().col1} @{item().col2} @{item().name} @{item().col5} and this should give output :

"value": "C0004\t1100\t\tAT020\t1102"

So that my activity is more generic and can be used for any table and columns, currently I have to hardcode the column names (id,col1,col2,name,col5)

Please suggest how can I achieve this.

Database Table Creation Statement:

create table test_account

(id varchar(10),

col1 varchar(10),

col2 varchar(10),

[name] varchar(10),

col5 varchar(10))

insert into test_account (id,col1,col2,[name],col5) VALUES

('C0004','1000','','AT020','1102'),

('C0004','1001','','AT020','1102'),

('C0004','1100','','AT020','1102')
Bhargava-MSFT 31,356 Reputation points Microsoft Employee Moderator

2023-04-04T22:13:24.3433333+00:00

Hello Arundhati Sen, Thanks for the additional details. My understanding is you want to evaluate the inner expression from the outer expression. Currently, this is not possible with ADF. I see the below feedback request submitted in adf feedback channel Appreciate if you could upvote & comment on this feedback, which would be open to the user community. This allows our product team to prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature. https://feedback.azure.com/d365community/idea/5c59743a-6d26-ec11-b6e6-000d3a4f032c https://feedback.azure.com/d365community/idea/f25cce9d-6c26-ec11-b6e6-000d3a4f032c

Your answer

Bhargava-MSFT 31,356 Reputation points Microsoft Employee Moderator

2023-03-30T17:28:11.8933333+00:00

Hello Arundhati Sen
Welcome to the MS Q&A platform.

Are you trying to pass the output of an activity as input to another activity in Azure Data Factory (ADF)? And are you trying to pass a string value that you constructed using the expression "@{item().id} @{item().col1} @{item().col2} @{item().name} @{item().col5}" as input to the second activity?

If so, what is the second activity, and what input format does it expect? Can you please provide more details on the activities you are working with and the pipeline flow would be helpful.
Bhargava-MSFT 31,356 Reputation points Microsoft Employee Moderator

2023-04-04T22:13:24.3433333+00:00

Hello Arundhati Sen, Thanks for the additional details. My understanding is you want to evaluate the inner expression from the outer expression. Currently, this is not possible with ADF. I see the below feedback request submitted in adf feedback channel Appreciate if you could upvote & comment on this feedback, which would be open to the user community. This allows our product team to prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature. https://feedback.azure.com/d365community/idea/5c59743a-6d26-ec11-b6e6-000d3a4f032c https://feedback.azure.com/d365community/idea/f25cce9d-6c26-ec11-b6e6-000d3a4f032c

Share via

how to dynamically transform one activity output and feed in another activity in ADF

Your answer