How can I use the Range function in the dynamic expression editor of the Data Flow Pagination Rules?

Question

With a Copy Activity pagination rules, you can use a Range function to generate a range of values for the offset like this : "paginationRules": { "AbsoluteUrl.{offset}": "RANGE:0:$.meta.total_count:1000" }

But this is not available with the Data Flow. I want to use Data Flow because I also want to take advantage of partitioning. How can I achieve the dynamic generation of the offset as in this example, and why is the range not available in Data Flow expression editor?

1. Copy Activity Pagination Rule
   
2. Data Flow Pagination Rule
   

Answer 1

Hi @Chaiwa ,

Thank you for posting query in Microsoft Q&A Platform.

Yes, In dataflows currently that option is not available. I am sharing this feedback to internal product team to consider as feature request. I will share updates on thread once I here back.

As work around you can consider Copy activity and load API response to ADLS as CSV or Parquet fies and then use that files are source in dataflows to apply transformations.

Hope this helps. Please let me know if any further queries.

---------
Please consider hitting Accept Answer button. Accepted answers help community as well.