Synapse - Rest API Returning Charset "UTF-8" (with the quotes)

SDHorita 1 Reputation point


I'm attempting to do a copy data activity from a Rest API data source, but it seems to be returning a header that says that the charset is "UTF-8" with the quotes, and Synapse is rejecting it.

"message": "Failure happened on 'Source' side. ErrorCode=UserErrorCharSetInvalidOrNotSupported,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The CharSet '\"utf-8\"' is invalid or not supported from Response.Headers.Content-Type.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.ArgumentException,Message='\"utf-8\"' is not a supported encoding name. For information on defining a custom encoding, see the documentation for the Encoding.RegisterProvider method.\r\nParameter name: name,Source=mscorlib,'"

I don't have control over the API that is being called, so I can't change what is being returned. I'm looking for a way to tell synapse to ignore the charset header and just assume that it's utf-8 regardless. Is that something that exists?

If not, is there something other than the copy activity in synapse that can ignore the headers from an API call and let me edit that header before it tries to process the data?


Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
3,098 questions
{count} votes

1 answer

Sort by: Most helpful
  1. KranthiPakala-MSFT 38,602 Reputation points Microsoft Employee

    Hello @SDHorita ,

    Thanks for the question and using MS Q&A platform.

    Unfortunately, there is no workaround as it is failing at the connector level which means, it is failing even before entering the ADF pipeline. Root cause is as you mentioned the double quote in encoding method which is not supported in .NET.

    Here is another thread where a similar issue has been discussed: Data Factory REST Dataset - support 'utf8' charset variant

    Ideally this should be handled on API end.

    Let us know if you have further questions.

    Thank you

    0 comments No comments