data factory foreach activity incorrect set variable

Question

Hello,in the the foreach activity, I have a set variable and a copy activity.the set variable assigns the variable to the correct text as follows:i.e. if source_entity is 'comment' then set the variable to 'LastEditDate' otherwise, set it to 'LastModifiedDate'

@if(equals(tolower(item().source_entity), 'comment'), 'LastEditDate', 'LastModifiedDate')

then in the source tab of the copy activity I am trying to use this variable but it does not seem to get assigned correctly.

@concat('SELECT * FROM ', item().source_entity, ' WHERE ',variables('v_modified_date_column'),' >= ',variables('v_last_modified_date_as_of'),'T00:00:00Z')

It looks like, this works fine if I tick the sequential property of the foreach activity but if it is not ticked then the variable does not seem to assign correctly...I do not want the sequential as there are alot of items to loop through...

Any thoughts?

thank you

Answer 1

Instead of setting the variable within a ForEach loop, why don't you try to handle the logic within the Copy Activity or Lookup activity itself dynamically? at least you will keep the variable setting logic scoped to the current iteration.

In your Copy Activity, try the following :

@concat('SELECT * FROM ', item().source_entity, ' WHERE ', 
if(equals(tolower(item().source_entity), 'comment'), 'LastEditDate', 'LastModifiedDate'),
' >= ', variables('v_last_modified_date_as_of'),'T00:00:00Z')