Pagination of REST API call with "Link" in Header and rel="next"

Kumar, Arun 296 Reputation points
2023-02-02T01:11:52.59+00:00

Hello,

I have a hard time doing pagination in ADF dataflow activity. Tried many of the below, nothing worked. My Link in Header is like below -

User's image

User's image

User's image

User's image

User's image

Any idea how to work on it?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,145 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,151 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Kumar, Arun 296 Reputation points
    2023-02-14T21:40:02.79+00:00

    @MartinJaffer-MSFT - i tried RFC 5988 also, did not work and it threw error. I solved the issue by creating a pyspark notebook to do looping of the pages. Don't think Azure have a mechanism to support these kind of scenarios.

    1 person found this answer helpful.