Azure Data Factory throws runtime error where as preview data returns data
I have a data factory which is failing for last 2 days. The pipeline has a copy activity which gets data from rest api and stores data to the data lake store. The preview data of the source dataset returns data but throws error while running the pipeline.
Failure happened on 'Source' side. ErrorCode=RestCallFailedWithClientError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Rest call failed with client error, status code 401 Unauthorized, please check your activity settings.
Request URL: https://XXXXXXX.com/Movements/XXXXXX?startDate=2022-11-09%2007:00:00&stopDate=2022-11-09%2008:00:00.
Response: ,Source=Microsoft.DataTransfer.ClientLibrary,'
Below is the input of the copy activity
{
"source": {
"type": "RestSource",
"httpRequestTimeout": "00:01:40",
"requestInterval": "00.00:00:00.010",
"requestMethod": "GET",
"additionalHeaders": {
"Authorization": "Basic YYYYYY"
},
"paginationRules": {
"supportRFC5988": "true"
}
},
"sink": {
"type": "JsonSink",
"storeSettings": {
"type": "AzureBlobFSWriteSettings"
},
"formatSettings": {
"type": "JsonWriteSettings"
}
},
"enableStaging": false
}