Getting "Found invalid data while decoding" error while running Copy Activity for REST API

Avnish Yadav 0 Reputation points
2024-05-09T08:22:18.68+00:00

Hi, I am trying to get data from REST API and trying to save it in CSV. However, each time i run the copy activity i am getting below error. Any help will be great appreciate it.

This REST API required a Header value as below to get uncompressed data

Accept-Encoding: identity

I have tried remove this but problem remain same.

{ "dataRead": 0, "dataWritten": 0, "filesWritten": 0, "sourcePeakConnections": 1, "sinkPeakConnections": 1, "rowsRead": 0, "rowsCopied": 0, "copyDuration": 9, "throughput": 0, "errors": [ { "Code": 23360, "Message": "Failure happened on 'Source' side. ErrorCode=RestResourceReadFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Fail to read from REST resource.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.IO.InvalidDataException,Message=Found invalid data while decoding.,Source=System,'", "EventType": 0, "Category": 5, "Data": { "FailureInitiator": "Source" }, "MsgId": null, "ExceptionType": null, "Source": null, "StackTrace": null, "InnerEventInfos": [] } ],

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,749 questions
{count} votes

1 answer

Sort by: Most helpful
  1. phemanth 6,710 Reputation points Microsoft Vendor
    2024-05-10T14:34:31.5333333+00:00

    @Avnish Yadav Here are some additional areas to investigate:

    Copy Activity Settings:

    • Format: Double-check the format settings in the Copy Activity source. Ensure it aligns with the format of the data returned by the REST API (e.g., JSON, CSV).
    • Compression: Verify that the compression setting in the Copy Activity source matches the compression used by the API (if any). You mentioned the API requires "Accept-Encoding: identity", so ensure compression is disabled in the Copy Activity source settings.
    • Data Type Mapping: Make sure the data type mapping between the source schema and the sink schema (CSV) is correct.

    Integration Runtime (IR):

    • Type: If you're using a Self-Hosted IR, ensure it has access to the internet and can reach the API endpoint.
    • Version: Consider using a newer version of the IR if you're using an older one.

    Logging:

    • Enable verbose logging for the Copy Activity to get more detailed information about the error. This might reveal additional clues about the decoding issue.

    Here are some additional resources that might be helpful:

    If you've already checked these areas and are still facing issues, consider sharing the following information for further assistance:

    • Copy Activity source configuration (format, compression settings).
    • REST API documentation (if available) or sample response.
    • Verbose logging output from the Copy Activity run.
    0 comments No comments