ADF Webactivity response XML accessing next page

sankon437 21 Reputation points
2024-06-05T13:03:05.3466667+00:00

XML response is giving next page url token, how to accomplish this is ADF using looping.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,923 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Amira Bedhiafi 18,026 Reputation points
    2024-06-05T16:03:59.0033333+00:00

    You can add a Web Activity to call the API and get the initial XML response and configure it with the necessary URL, method, and headers.

    Add an Azure Function or a Data Flow activity to parse the XML response and extract the next page URL where you can use logic to extract the next page URL token from the XML response.

    You can use a Set Variable activity to store the next page URL extracted in the previous step.

    Add an Until activity to loop through the pages. Set the condition for the loop to continue until there is no next page URL and inside of it, you need to add a sequence of activities to fetch the next page and process the response.

    Inside the Until activity, add another Web Activity to call the API with the next page URL stored in the variable.

    Parse the response in the same manner as the initial response to extract data and the next page URL and update the variable with the new next page URL.

    1 person found this answer helpful.