Getting "Found invalid data while decoding" error while running Copy Activity for REST API

Question

Hi, I am trying to get data from REST API and trying to save it in CSV. However, each time i run the copy activity i am getting below error. Any help will be great appreciate it.

This REST API required a Header value as below to get uncompressed data

Accept-Encoding: identity

I have tried remove this but problem remain same.

{ "dataRead": 0, "dataWritten": 0, "filesWritten": 0, "sourcePeakConnections": 1, "sinkPeakConnections": 1, "rowsRead": 0, "rowsCopied": 0, "copyDuration": 9, "throughput": 0, "errors": [ { "Code": 23360, "Message": "Failure happened on 'Source' side. ErrorCode=RestResourceReadFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Fail to read from REST resource.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.IO.InvalidDataException,Message=Found invalid data while decoding.,Source=System,'", "EventType": 0, "Category": 5, "Data": { "FailureInitiator": "Source" }, "MsgId": null, "ExceptionType": null, "Source": null, "StackTrace": null, "InnerEventInfos": [] } ],

Answer 1

@Avnish Yadav Here are some additional areas to investigate:

Copy Activity Settings:

• Format: Double-check the format settings in the Copy Activity source. Ensure it aligns with the format of the data returned by the REST API (e.g., JSON, CSV).
• Compression: Verify that the compression setting in the Copy Activity source matches the compression used by the API (if any). You mentioned the API requires "Accept-Encoding: identity", so ensure compression is disabled in the Copy Activity source settings.
• Data Type Mapping: Make sure the data type mapping between the source schema and the sink schema (CSV) is correct.

Integration Runtime (IR):

• Type: If you're using a Self-Hosted IR, ensure it has access to the internet and can reach the API endpoint.
• Version: Consider using a newer version of the IR if you're using an older one.

Logging:

• Enable verbose logging for the Copy Activity to get more detailed information about the error. This might reveal additional clues about the decoding issue.

Here are some additional resources that might be helpful:

• Troubleshoot copy activity errors: https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-overview
• Copy activity data format properties: https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-overview

If you've already checked these areas and are still facing issues, consider sharing the following information for further assistance:

• Copy Activity source configuration (format, compression settings).
• REST API documentation (if available) or sample response.
• Verbose logging output from the Copy Activity run.