Getting "Found invalid data while decoding" error while running Copy Activity for REST API

Avnish Yadav 0

Hi, I am trying to get data from REST API and trying to save it in CSV. However, each time i run the copy activity i am getting below error. Any help will be great appreciate it.

This REST API required a Header value as below to get uncompressed data

Accept-Encoding: identity

I have tried remove this but problem remain same.

{ "dataRead": 0, "dataWritten": 0, "filesWritten": 0, "sourcePeakConnections": 1, "sinkPeakConnections": 1, "rowsRead": 0, "rowsCopied": 0, "copyDuration": 9, "throughput": 0, "errors": [ { "Code": 23360, "Message": "Failure happened on 'Source' side. ErrorCode=RestResourceReadFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Fail to read from REST resource.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.IO.InvalidDataException,Message=Found invalid data while decoding.,Source=System,'", "EventType": 0, "Category": 5, "Data": { "FailureInitiator": "Source" }, "MsgId": null, "ExceptionType": null, "Source": null, "StackTrace": null, "InnerEventInfos": [] } ],

phemanth 6,550 Reputation points Microsoft Vendor

2024-05-09T13:18:23.7033333+00:00
@Avnish Yadav

Welcome to Microsoft Q&A forum and thanks for reaching out here.

The error code 23360 indicates that there was a failure on the source side while reading from the REST resource. Let’s explore some possible solutions to address this issue:

Check the REST API Endpoint:

Ensure that the REST API endpoint is accessible and responding correctly.

Verify that the URL you are using is correct and that the API is up and running.

Headers and Authentication:

You mentioned that the REST API requires an Accept-Encoding: identity header. Make sure you are including this header in your request.

Additionally, check if there are any other required headers or authentication tokens needed for successful communication with the API.

Network Connectivity:

The error message also mentions that the connection attempt failed. Ensure that your network connectivity is stable and that there are no firewall rules blocking the communication.

If the REST API is hosted on a specific IP address, verify that your system can reach that IP.

SSL/TLS Configuration:

Sometimes SSL/TLS configurations can cause issues. Ensure that your system supports the same SSL/TLS version as the REST API server.

If the REST API server has recently updated its cipher suites, make sure your system is compatible with the new configuration.

Character Set Issues:

Although not directly related to error code 23360, it’s worth checking if the response from the REST API contains any invalid characters or character sets. Ensure that your code can handle the response correctly

Retry Mechanism:

Implement a retry mechanism in your code to handle transient errors. Sometimes the REST API might be temporarily unavailable, and retrying the request after a short delay could resolve the issue.

Hope this helps. Do let us know if you any further queries.
Avnish Yadav 0 Reputation points

2024-05-09T21:54:23.5433333+00:00

Hi, Thank you for the reply. I have checked all of these areas. To verify the same, i have done same REST API call through HTTP Connector (WebActivity) in Data Factory and got the result. however when i am trying REST API Connector through Copy Activity it gives me this error.

Still the same error:

{ "dataRead": 0, "dataWritten": 0, "filesWritten": 0, "sourcePeakConnections": 1, "sinkPeakConnections": 1, "rowsRead": 0, "copyDuration": 12, "throughput": 0, "errors": [ { "Code": 23360, "Message": "Failure happened on 'Source' side. ErrorCode=RestResourceReadFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Fail to read from REST resource.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.IO.InvalidDataException,Message=Found invalid data while decoding.,Source=System,'", "EventType": 0, "Category": 5, "Data": { "FailureInitiator": "Source" }, "MsgId": null, "ExceptionType": null, "Source": null, "StackTrace": null, "InnerEventInfos": [] } ],
phemanth 6,550 Reputation points Microsoft Vendor

2024-05-10T14:34:31.5333333+00:00
@Avnish Yadav Here are some additional areas to investigate:

Copy Activity Settings:

Format: Double-check the format settings in the Copy Activity source. Ensure it aligns with the format of the data returned by the REST API (e.g., JSON, CSV).

Compression: Verify that the compression setting in the Copy Activity source matches the compression used by the API (if any). You mentioned the API requires "Accept-Encoding: identity", so ensure compression is disabled in the Copy Activity source settings.

Data Type Mapping: Make sure the data type mapping between the source schema and the sink schema (CSV) is correct.

Integration Runtime (IR):

Type: If you're using a Self-Hosted IR, ensure it has access to the internet and can reach the API endpoint.

Version: Consider using a newer version of the IR if you're using an older one.

Logging:

Enable verbose logging for the Copy Activity to get more detailed information about the error. This might reveal additional clues about the decoding issue.

Here are some additional resources that might be helpful:

Troubleshoot copy activity errors: https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-overview

Copy activity data format properties: https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-overview

If you've already checked these areas and are still facing issues, consider sharing the following information for further assistance:

Copy Activity source configuration (format, compression settings).

REST API documentation (if available) or sample response.

Verbose logging output from the Copy Activity run.
phemanth 6,550 Reputation points Microsoft Vendor

2024-05-13T06:02:01.6066667+00:00

@Avnish Yadav We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Share via

Getting "Found invalid data while decoding" error while running Copy Activity for REST API