How to use pagination in Azure Data factory with REST API using continuationToken

Kanukunta, Suparna R 5 Reputation points
2023-08-17T01:33:26.9266667+00:00

I am trying the solution provided in below link to retrieve the data from "Pipeline Runs - Query By Factory" from the Azure REST API using Azure Data factory.

https://learn.microsoft.com/en-us/answers/questions/569319/how-to-use-pagination-in-azure-data-factory-with-r

When the pipelines passes continuation token in Body of copy activity, i get below error

Request Body in Copy Activity:

{
lastUpdatedAfter:@{concat('"',formatDateTime(string(
	convertTimeZone(activity('lkup_max_logdt').output.firstRow.maxdt, 'Central Standard Time','UTC')), 'yyyy-MM-dd hh:mm:ss'),'"')},
lastUpdatedBefore:@{concat('"',formatDateTime(pipeline().TriggerTime, 'yyyy-MM-dd hh:mm:ss'),'"')},
continuationToken:@{variables('continuationToken')}
}

Error:

Failure happened on 'Source' side. ErrorCode=RestCallFailedWithClientError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Rest call failed with client error, status code 400 BadRequest, please check your activity settings.
Request URL: https://management.azure.com/subscriptions/cbfd2115-3a52-490a-92fc-abbf03d1632e/resourceGroups/era-dandi-prd/providers/Microsoft.DataFactory/factories/dandi-adf-call-prd/queryPipelineRuns?api-version=2018-06-01.
Response: {
  "errors": {
    "": [
      "JsonToken EndArray is not valid for closing JsonType Object. Path '', line 4, position 12230."
    ],
    "continuationToken": [
      "Unexpected character encountered while parsing value: [. Path 'continuationToken', line 4, position 19."
    ]
  },
  "type": "https://tools.ietf.org/html/rfc7231#section-6.5.1",
  "title": "One or more validation errors occurred.",
  "status": 400,
  "traceId": "00-96cc804191b5b267cdb3f345671ef3b8-06f7e2b2a4f5cf42-01"
},Source=Microsoft.DataTransfer.ClientLibrary,'

How can i resolve this issue ?

Thank you

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,639 questions
{count} vote

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.