Azure Data-factory: pagination with graph api as REST source doesn't seem to work?

Raf Cox 26 Reputation points
2020-12-07T12:49:28.93+00:00

Hi,
(not an expert in this area, so ...)
I try to copy data using Azure Data Factory from Microsoft Graph API (e.g. https://graph.microsoft.com/beta/users) to azure blob-storage.
copy works fine for 1 page, but it doesn't retrieve the next pages;

I have a pipeline with 1 activity: copy data
source is configured as REST api and I specified the pagination parameter: AbsoluteURL=$."@odata.nextLink"

sink is an Azure Blob storage

when I trigger the pipeline, it reads the data for the first page, writes it to the blob-storage and it then stops...

Am I missing something? some extra config required?

thanks!
Raf

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,545 questions
Microsoft Entra ID
Microsoft Entra ID
A Microsoft Entra identity service that provides identity management and access control capabilities. Replaces Azure Active Directory.
19,474 questions
0 comments No comments
{count} vote

Accepted answer
  1. Saurabh Sharma 23,676 Reputation points Microsoft Employee
    2020-12-09T00:56:31.587+00:00

    @Raf Cox Thanks for using Microsoft Q&A.

    You need to pass AbsoluteURL as $['@odata.nextLink'] to make it work. I have tested in my environment and this works for me while fetching Microsoft Graph API. (Please find the screenshot).

    46424-image.png

    ----------

    Please do not forget to "Accept the answer" wherever the information provided helps you to help others in the community.

    5 people found this answer helpful.

0 additional answers

Sort by: Most helpful