Azure Data Factory throws runtime error where as preview data returns data

Prasant Nanda 1 Reputation point
2022-11-09T13:34:31.367+00:00

I have a data factory which is failing for last 2 days. The pipeline has a copy activity which gets data from rest api and stores data to the data lake store. The preview data of the source dataset returns data but throws error while running the pipeline.

Failure happened on 'Source' side. ErrorCode=RestCallFailedWithClientError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Rest call failed with client error, status code 401 Unauthorized, please check your activity settings.
Request URL: https://XXXXXXX.com/Movements/XXXXXX?startDate=2022-11-09%2007:00:00&stopDate=2022-11-09%2008:00:00.
Response: ,Source=Microsoft.DataTransfer.ClientLibrary,'

Below is the input of the copy activity

{
"source": {
"type": "RestSource",
"httpRequestTimeout": "00:01:40",
"requestInterval": "00.00:00:00.010",
"requestMethod": "GET",
"additionalHeaders": {
"Authorization": "Basic YYYYYY"
},
"paginationRules": {
"supportRFC5988": "true"
}
},
"sink": {
"type": "JsonSink",
"storeSettings": {
"type": "AzureBlobFSWriteSettings"
},
"formatSettings": {
"type": "JsonWriteSettings"
}
},
"enableStaging": false
}

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,596 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.