ADF Resource Manager ContinuationToken not working

Question

ADF Resource Manager ContinuationToken not working

Tommy Tam 0

I'm using the ADF Resource Manager as per this documentation. I have managed to make the first Post API call to return the first 100 results as well as the continuationToken, but there is no documentation on how to use this to retrieve the next page of results.

The format of the continutationToken is as follows:

"continuationToken": "[
  {
    "token": "token_value",
    "range": {
      "min": "05C1EBFB6DDB0C",
      "max": "FF"
    }
  }
]"

In the next API call, I have tried putting in this entire continutationToken like this:

"continutationToken" : "@{json(activity('List All Pipeline Runs').output.continuationToken)}"

But it gives me this error (same even if I remove json() from above):

{
  "errors": {
    "continutationToken": [
      "After parsing a value an unexpected character was encountered: t. Path 'continutationToken', line 13, position 28."
    ]
  },
  "type": "https://tools.ietf.org/html/rfc9110#section-15.5.1",
  "title": "One or more validation errors occurred.",
  "status": 400,
  "traceId": "00-e9b40a781284a7b78418ab3b5ee889e5-4f32017ab1886866-01"
}

enter image description here

I have also tried to feed in just the token_value into the next API call like this:

"continutationToken" : "@{json(activity('List All Pipeline Runs').output.continuationToken)[0].token}"

This ran successfully, but the results returned are identical to my initial API call and so not giving me the next page of results still.

Venkat Reddy Navari 1,700 Reputation points Microsoft External Staff Moderator

2025-05-06T11:42:53.81+00:00
Hi@Tommy Tam You're encountering the following error:

“After parsing a value an unexpected character was encountered: t. Path 'continuationToken', line 13, position 28.”

This typically means that the continuationToken is being passed as a string instead of as a native JSON object or array.

You're using:

"continuationToken": "@{json(activity('List All Pipeline Runs').output.continuationToken)}"

Change it to:

"continuationToken": @{activity('List All Pipeline Runs').output.continuationToken}

This ensures the token is passed as structured JSON, not a string.

Token value alone isn’t enough

"continuationToken": "@{activity('List All Pipeline Runs').output.continuationToken[0].token}"

This might succeed but will return the same page of results. That's because the full continuationToken (including min and max range) is required for paging.

How to confirm the continuationToken structure

Check the output of the “List All Pipeline Runs” activity.

Look for a continuationToken property like this:

[ { "token": "abc123...", "range": { "min": "05C1...", "max": "FF" } } ]

If it's shown as a string with escaped characters (e.g., "[{\"token\":\"...\"}]"), that means it's being passed incorrectly — and will trigger a parsing error.

The error: “unexpected character was encountered: t…”

Suggests the continuationToken was passed as a string (starting with "token"), rather than a JSON object. This matches the issue exactly — removing quotes and passing the array directly should resolve it.

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.
Tommy Tam 0 Reputation points

2025-05-06T11:56:29.61+00:00

@Venkat Reddy Navari

Thanks for your suggestion, unfortunately updating the expression builder to

has returned the same results as the initial API run, in fact I've noticed the same behaviour when I put in any value I want in the continuationToken
Venkat Reddy Navari 1,700 Reputation points Microsoft External Staff Moderator

2025-05-07T09:50:16.7233333+00:00
@Tommy Tam

You’ve inserted the correct dynamic reference using @{activity('List All Pipeline Runs').output.continuationToken}

You’re getting the same results regardless of the continuationToken’s actual value

Manual input like "anyvaluehere" also returns the same initial 100 results

it strongly suggests the API is not utilizing the continuationToken value to fetch subsequent pages.

Review the Time Range Filters: Double-check the lastUpdatedAfter and lastUpdatedBefore fields in your request body. Currently, they’re set to a 5-year range (lastUpdatedAfter as 5 years ago and lastUpdatedBefore as now). Ensure this range isn’t causing the API to return the same set of results, potentially overriding the continuationToken logic.

Test the API Outside ADF: Try calling the API using Postman or a REST client with the full continuationToken object in the request body. Here’s the structure again:
[ { "token": "your_token_here", "range": { "min": "05C1EBFB6DDB0C", "max": "FF" } } ]
If this returns the next page, then the issue lies with how the Data Factory pipeline is forming or handling the call. If it doesn’t, it’s likely a backend pagination issue.

Debug the Token in ADF: To confirm the continuationToken value being passed, you can add a Set Variable activity after the List All Pipeline Runs activity to capture and log the continuationToken. This will help verify that the token structure matches the expected format.

Raise a Support Request: If the API isn’t honoring the continuationToken regardless of usage, please consider raising a support request with Microsoft, as this might be an issue with how pagination is implemented on the server side.

Workaround with Manual Pagination: As a workaround, you can implement manual pagination using the lastUpdatedAfter and lastUpdatedBefore fields. After each API call, extract the timestamp of the latest pipeline run from the results (e.g., using the lastUpdated field of the last record), then set lastUpdatedAfter to that timestamp for the next call. This will progressively shift the time window to fetch the next set of records.

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.
Venkat Reddy Navari 1,700 Reputation points Microsoft External Staff Moderator

2025-05-08T10:18:30.39+00:00

@Tommy Tam We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Tommy Tam 0 Reputation points

2025-05-08T10:33:11.4733333+00:00

@Venkat Reddy Navari There is no resolution unfortunately. The first API has the same time range filters, so doesn't explain why it wouldn't work for the next API. Even if it works in Postman, that still doesn't help because this needs to work within ADF. Passing it through as a variable only told me what I already knew, the format is fine, and even if not we were already applying the json function to it anyway. Haven't got subscription permission to raise a support request. And finally the pagination isn't a good enough workaround as that would take too long. I am just surprised that Microsoft have provided a continuationToken that clearly doens't work, which is very frustrating.
Max Kirkman 20 Reputation points

2025-05-12T22:31:11.6833333+00:00
I experienced the same issue. It wasn't an issue until just recently.

The continuationToken begins with the token and range metadata, while subsequent API calls return only the continuationToken token value.

Try this:

@if( contains(activity('SourceContinuationToken').output.continuationToken, 'token'), json(activity('SourceContinuationToken').output.continuationToken)[0].token, activity('SourceContinuationToken').output.continuationToken )
Tommy Tam 0 Reputation points

2025-05-13T07:59:04.1133333+00:00
Thanks @Max Kirkman , but my issue is different where it's not whether the continuationToken contains "token" or not. Because I've already tried to include and exclude both the range metadata, my issue is that it still returns the same page of results as the first API.

Furthermore in your code above, you actually need to concatenate

json(activity('SourceContinuationToken').output.continuationToken)[0].token

with double quotes because that's a string, and also you need to add

json()

to the else statement because that's an object. Like this:

@if(contains(activity('List All Pipeline Runs').output.continuationToken, 'token'), concat('"', json(activity('List All Pipeline Runs').output.continuationToken)[0].token, '"'), json(activity('List All Pipeline Runs').output.continuationToken))
Shraddha Pore 15 Reputation points Microsoft External Staff Moderator

2025-05-13T09:34:03.8733333+00:00
@Tommy Tam Implementing Pagination in ADF

Since ADF's Copy Activity doesn't support pagination rules for continuation tokens in the request body, you need to implement pagination manually using a loop. Here's how you can do it:Microsoft Learn+1Microsoft Learn+1

Initial API Call

Use a Web Activity to make the initial API call and retrieve the first page of results along with the continuation token.

Store the Continuation Token

Use a Set Variable Activity to store the continuation token from the response.

Loop to Fetch Subsequent Pages

Use a ForEach Activity to iterate over the continuation tokens. In each iteration:

Set the continuation token variable to the current token.

Make the API call with the continuation token in the request body.

Process the results.

Check if the response contains a continuation token. If it does, continue to the next iteration; if not, exit the loop.
Tommy Tam 0 Reputation points

2025-05-13T09:51:03.65+00:00

@Shraddha Pore sorry but that is not my issue here, I am not attempting to use pagination, and I know how to use the continuationToken. The issue here is that despite using the continuationToken (whether it includes the range metadata or not, trust me I've tried both) to get the next page of results, the returned results remain unchanged and is the same as my first page of results.
Max Kirkman 20 Reputation points

2025-05-13T18:31:04.78+00:00

@Tommy Tam, my code-block works and follows a pattern similar to @Shraddha Pore's. I use an Until activity to iterate until the continuationToken is null, ensuring all data is retrieved.
Tommy Tam 0 Reputation points

2025-05-13T18:42:21.76+00:00

@Max Kirkman My iteration will never end because the result returns the same continuationToken as the first API call, as I've mentioned before I've tried

and

and even this which should never work

but all 3 gives me the same result as the first API call, here's a screenshot of Notepad++ to prove it:
Max Kirkman 20 Reputation points

2025-05-13T20:41:25.8533333+00:00

Here is the outer Until loop, which is used to iterate until no continuation tokens are available:

The condition for the Until loop to terminate:

Activity 1: Set Variable

Activity 2: Copy Activity

Activity 3: Web Activity

Activity 4: Set Variable

Activity 5 (hidden in the first image): Set Variable

**(If Activity 4 fails (On Failure)),*

, where the null value for the pipelineRunsContinuationToken variable causes the Until loop to terminate.
Tommy Tam 0 Reputation points

2025-05-14T00:08:35.0766667+00:00

@Anonymous again I am not having problem setting up the loop, I fully understand the logic, my problem is whatever value I put into the continationToken, I am getting the same result as the first API. Just try this would you expect it to work?
Shraddha Pore 15 Reputation points Microsoft External Staff Moderator

2025-05-14T05:35:17.9266667+00:00
@Tommy Tam, Thanks for the response. Possible reasons for the error are as below

The continuation token isn’t being used correctly: The API might not be expecting the token in the request body. Some APIs require it to be sent in a different way—like in the headers, as a query parameter, or within a specific JSON field (like nextLink or nextPageToken).

The token is context-specific: These tokens are usually tied to the specific state of the previous API call. If we’re using a static or randomly generated token, it probably won’t work.

The API might not use continuation tokens at all: Some APIs paginate based on timestamps, record IDs, or offsets. In these cases, there wouldn’t be a continuation token—just a different kind of marker.

First look at the actual response from the first API call. You need to check if it includes anything like continuationToken, nextLink, or nextPageToken.

Also need to make sure you’re extracting that token correctly and passing it into the next request—wherever the API expects it (query, header, or body).

Ensure the API expects the token in the body If the API uses a full nextLink, we might just need to call that URL directly for the next page.
Tommy Tam 0 Reputation points

2025-05-14T08:06:19.2733333+00:00

@Shraddha Pore That's why I'm raising it in this forum because this API is for Azure Data Factory which is by Microsoft, I have followed this documentation to do exactly what I should. Problem is it's not working and everyone else keeps telling me to pass it as a variable or place it in a Until activity and loop until the continuationToken is null, but that's not the problem I'm having. The issue is the continuationToken is returning the same result as my first API call which isn't right!

Your answer

Venkat Reddy Navari 1,700 Reputation points Microsoft External Staff Moderator

2025-05-06T11:42:53.81+00:00

Hi@Tommy Tam You're encountering the following error:

“After parsing a value an unexpected character was encountered: t. Path 'continuationToken', line 13, position 28.”

This typically means that the continuationToken is being passed as a string instead of as a native JSON object or array.

You're using:

"continuationToken": "@{json(activity('List All Pipeline Runs').output.continuationToken)}"

Change it to:

"continuationToken": @{activity('List All Pipeline Runs').output.continuationToken}

This ensures the token is passed as structured JSON, not a string.

Token value alone isn’t enough

"continuationToken": "@{activity('List All Pipeline Runs').output.continuationToken[0].token}"

This might succeed but will return the same page of results. That's because the full continuationToken (including min and max range) is required for paging.

How to confirm the continuationToken structure

Check the output of the “List All Pipeline Runs” activity.

Look for a continuationToken property like this:

[ { "token": "abc123...", "range": { "min": "05C1...", "max": "FF" } } ]

If it's shown as a string with escaped characters (e.g., "[{\"token\":\"...\"}]"), that means it's being passed incorrectly — and will trigger a parsing error.

The error: “unexpected character was encountered: t…”

Suggests the continuationToken was passed as a string (starting with "token"), rather than a JSON object. This matches the issue exactly — removing quotes and passing the array directly should resolve it.

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.
Tommy Tam 0 Reputation points

2025-05-06T11:56:29.61+00:00

@Venkat Reddy Navari

Thanks for your suggestion, unfortunately updating the expression builder to

has returned the same results as the initial API run, in fact I've noticed the same behaviour when I put in any value I want in the continuationToken
Venkat Reddy Navari 1,700 Reputation points Microsoft External Staff Moderator

2025-05-08T10:18:30.39+00:00

@Tommy Tam We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Tommy Tam 0 Reputation points

2025-05-08T10:33:11.4733333+00:00

@Venkat Reddy Navari There is no resolution unfortunately. The first API has the same time range filters, so doesn't explain why it wouldn't work for the next API. Even if it works in Postman, that still doesn't help because this needs to work within ADF. Passing it through as a variable only told me what I already knew, the format is fine, and even if not we were already applying the json function to it anyway. Haven't got subscription permission to raise a support request. And finally the pagination isn't a good enough workaround as that would take too long. I am just surprised that Microsoft have provided a continuationToken that clearly doens't work, which is very frustrating.
Max Kirkman 20 Reputation points

2025-05-12T22:31:11.6833333+00:00

I experienced the same issue. It wasn't an issue until just recently.

The continuationToken begins with the token and range metadata, while subsequent API calls return only the continuationToken token value.

Try this:

@if( contains(activity('SourceContinuationToken').output.continuationToken, 'token'), json(activity('SourceContinuationToken').output.continuationToken)[0].token, activity('SourceContinuationToken').output.continuationToken )
Tommy Tam 0 Reputation points

2025-05-13T07:59:04.1133333+00:00

Thanks @Max Kirkman , but my issue is different where it's not whether the continuationToken contains "token" or not. Because I've already tried to include and exclude both the range metadata, my issue is that it still returns the same page of results as the first API.

Furthermore in your code above, you actually need to concatenate

json(activity('SourceContinuationToken').output.continuationToken)[0].token

with double quotes because that's a string, and also you need to add

json()

to the else statement because that's an object. Like this:

@if(contains(activity('List All Pipeline Runs').output.continuationToken, 'token'), concat('"', json(activity('List All Pipeline Runs').output.continuationToken)[0].token, '"'), json(activity('List All Pipeline Runs').output.continuationToken))
Shraddha Pore 15 Reputation points Microsoft External Staff Moderator

2025-05-13T09:34:03.8733333+00:00

@Tommy Tam Implementing Pagination in ADF

Since ADF's Copy Activity doesn't support pagination rules for continuation tokens in the request body, you need to implement pagination manually using a loop. Here's how you can do it:Microsoft Learn+1Microsoft Learn+1

Initial API Call

Use a Web Activity to make the initial API call and retrieve the first page of results along with the continuation token.

Store the Continuation Token

Use a Set Variable Activity to store the continuation token from the response.

Loop to Fetch Subsequent Pages

Use a ForEach Activity to iterate over the continuation tokens. In each iteration:

Set the continuation token variable to the current token.

Make the API call with the continuation token in the request body.

Process the results.

Check if the response contains a continuation token. If it does, continue to the next iteration; if not, exit the loop.
Tommy Tam 0 Reputation points

2025-05-13T09:51:03.65+00:00

@Shraddha Pore sorry but that is not my issue here, I am not attempting to use pagination, and I know how to use the continuationToken. The issue here is that despite using the continuationToken (whether it includes the range metadata or not, trust me I've tried both) to get the next page of results, the returned results remain unchanged and is the same as my first page of results.
Max Kirkman 20 Reputation points

2025-05-13T18:31:04.78+00:00

@Tommy Tam, my code-block works and follows a pattern similar to @Shraddha Pore's. I use an Until activity to iterate until the continuationToken is null, ensuring all data is retrieved.
Tommy Tam 0 Reputation points

2025-05-13T18:42:21.76+00:00

@Max Kirkman My iteration will never end because the result returns the same continuationToken as the first API call, as I've mentioned before I've tried

and

and even this which should never work

but all 3 gives me the same result as the first API call, here's a screenshot of Notepad++ to prove it:
Max Kirkman 20 Reputation points

2025-05-13T20:41:25.8533333+00:00

Here is the outer Until loop, which is used to iterate until no continuation tokens are available:

The condition for the Until loop to terminate:

Activity 1: Set Variable

Activity 2: Copy Activity

Activity 3: Web Activity

Activity 4: Set Variable

Activity 5 (hidden in the first image): Set Variable

**(If Activity 4 fails (On Failure)),*

, where the null value for the pipelineRunsContinuationToken variable causes the Until loop to terminate.
Tommy Tam 0 Reputation points

2025-05-14T00:08:35.0766667+00:00

@Anonymous again I am not having problem setting up the loop, I fully understand the logic, my problem is whatever value I put into the continationToken, I am getting the same result as the first API. Just try this would you expect it to work?
Shraddha Pore 15 Reputation points Microsoft External Staff Moderator

2025-05-14T05:35:17.9266667+00:00

@Tommy Tam, Thanks for the response. Possible reasons for the error are as below

The continuation token isn’t being used correctly: The API might not be expecting the token in the request body. Some APIs require it to be sent in a different way—like in the headers, as a query parameter, or within a specific JSON field (like nextLink or nextPageToken).

The token is context-specific: These tokens are usually tied to the specific state of the previous API call. If we’re using a static or randomly generated token, it probably won’t work.

The API might not use continuation tokens at all: Some APIs paginate based on timestamps, record IDs, or offsets. In these cases, there wouldn’t be a continuation token—just a different kind of marker.

First look at the actual response from the first API call. You need to check if it includes anything like continuationToken, nextLink, or nextPageToken.

Also need to make sure you’re extracting that token correctly and passing it into the next request—wherever the API expects it (query, header, or body).

Ensure the API expects the token in the body If the API uses a full nextLink, we might just need to call that URL directly for the next page.
Tommy Tam 0 Reputation points

2025-05-14T08:06:19.2733333+00:00

@Shraddha Pore That's why I'm raising it in this forum because this API is for Azure Data Factory which is by Microsoft, I have followed this documentation to do exactly what I should. Problem is it's not working and everyone else keeps telling me to pass it as a variable or place it in a Until activity and loop until the continuationToken is null, but that's not the problem I'm having. The issue is the continuationToken is returning the same result as my first API call which isn't right!

Share via

ADF Resource Manager ContinuationToken not working

Your answer