Azure Data-factory: pagination with graph api as REST source doesn't seem to work?

Raf Cox 36 Reputation points
2020-12-07T12:49:28.93+00:00

Hi,
(not an expert in this area, so ...)
I try to copy data using Azure Data Factory from Microsoft Graph API (e.g. https://graph.microsoft.com/beta/users) to azure blob-storage.
copy works fine for 1 page, but it doesn't retrieve the next pages;

I have a pipeline with 1 activity: copy data
source is configured as REST api and I specified the pagination parameter: AbsoluteURL=$."@odata.nextLink"

sink is an Azure Blob storage

when I trigger the pipeline, it reads the data for the first page, writes it to the blob-storage and it then stops...

Am I missing something? some extra config required?

thanks!
Raf

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
Microsoft Security Microsoft Entra Microsoft Entra ID
0 comments No comments
{count} vote

Accepted answer
  1. Saurabh Sharma 23,846 Reputation points Microsoft Employee Moderator
    2020-12-09T00:56:31.587+00:00

    @Raf Cox Thanks for using Microsoft Q&A.

    You need to pass AbsoluteURL as $['@odata.nextLink'] to make it work. I have tested in my environment and this works for me while fetching Microsoft Graph API. (Please find the screenshot).

    46424-image.png

    ----------

    Please do not forget to "Accept the answer" wherever the information provided helps you to help others in the community.

    6 people found this answer helpful.

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.