Azure Data Factory condition in Foreach activity

Question

I have a requirement to load API data into cosmos db . So need to loop all the api pages for given period and load in cosmos db.

my pipeline has a web activity to get the api response first . then passing the page_count to foreach activity to loop through.

When there is data in API response the output looks below .

When API doesnt have data the output looks below.

I have written below condition in foreach activity for looping . Since the web activity output not same in both conditions , the below code not working .any suggestion please.

@Rover (1,if(contains(activity('WEB_GET_NUMBEROFPAGES').output.Response,'No data found'),1,activity('WEB_GET_NUMBEROFPAGES').output.ameta[0].page_count))

Answer 1

Hi @Abdul Kalam Mulla ,
Thankyou for the detailed description about your ask. In the above expression , one or the other property would always be missing , either 'Response' or 'ameta[]' . So, we need to first check the available properties , before using them in the expression.

Here is what you need to do:
1. Create a variable var1 of type 'String'. Use Set variable activity after the Web activity to store the output of Web activity. Use this in the value of set variable activity: @{activity('Web1').output} or @string(activity('Web1').output)

2. Use If Condition activity with expression : @contains(variables('var1'),'No data') .
a. Inside True condition block: Use Execute Pipeline activity and create a new pipeline . In the new pipeline, Use For Each activity and have this
expression as the Items value: @range(1,1)

b. Inside False condition block: Use Execute Pipeline activity and create a new pipeline. In the new pipeline , Create a parameter named page_count and go back to the False condition block and provide activity('WEB_GET_NUMBEROFPAGES').output.ameta[0].page_count as the value for page_count variable. Use For Each activity in the new pipeline called inside False block and provide this as items value: @range(1,int(pipeline().parameters.page_count))

Note: In the images, I have used lookup instead of web activity to reproduce your scenario.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you.
  Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Answer 2

Hi @Abdul Kalam Mulla ,

Thankyou for using Microsoft Q&A platform and thanks for posting your query here.

As I understand your ask , it seems you are trying to loop through each of the responses of Web Activity inside ForEach activity based on the condition:
If Webactivity output contains 'No data found', items value should be @Rover (1,1) else @Rover (1,page_count) . However, it's not working as expected. Please correct me if my understanding is incorrect.

Please try the following expression: @range(1,if(contains(activity('WEB_GET_NUMBEROFPAGES').output.Response.data[0],'No data found'),1,activity('WEB_GET_NUMBEROFPAGES').output.ameta[0].page_count)) as 'No data found' is part of data[] array , we need to mention that to check if that string is part of the output or not.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you.
  Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators