ADF is failing for special characters in copy activity

Question

ADF is failing for special characters in copy activity

Raja 86

Hi Team,
We've a ADF pipeline written which pulls information from 'Salesforce' as source in the form of 'select query' and saves the records in the form of files in ADLS container.
It's been working perfectly for long time, but recently it started failing with error,

Operation on target Save data in container failed: Failure happened on 'Source' side. ErrorCode=UserErrorOdbcOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ERROR [HY000] [Microsoft][Salesforce] (209) JSON parsing error: The surrogate pair in string is invalid.,Source=Microsoft.DataTransfer.ClientLibrary.Odbc.OdbcConnector,''Type=System.Data.Odbc.OdbcException,Message=ERROR [HY000] [Microsoft][Salesforce] (209) JSON parsing error: The surrogate pair in string is invalid.,Source=Microsoft Salesforce ODBC Driver,'

We haven't made any changes in the ADF pipeline and neither on the salesforce side anything has changed.
What we noticed is, there some data coming which has some special characters due to which it is failing.

This happens at the copy activity. We have a very simple copy activity,

We are sure that this problem we are seeing is due to special characters in the data. These special characters are valid and expected to be there, we cannot remove them.
What can be done here? how to fix this issue?

Any help is highly appreciated.

Thanks

NS 5 Reputation points

2024-01-09T02:59:31.7766667+00:00

@Raja I am wondering if you received any suggestions from Microsoft. We recently migrated on Azure and having really hard time with the special characters, carriage returns. They are in backend descriotion of SF columns and pushing all the IDs when we are copying the data from SF. Opened the tickes with Microsoft but they are just taking forever rather solving the problem.
Tahirah Ahmad 0 Reputation points

2024-07-19T01:21:29.41+00:00

did you figure out an answer? Also ran into this issue

1 answer

Your answer

NS 5 Reputation points

2024-01-09T02:59:31.7766667+00:00

@Raja I am wondering if you received any suggestions from Microsoft. We recently migrated on Azure and having really hard time with the special characters, carriage returns. They are in backend descriotion of SF columns and pushing all the IDs when we are copying the data from SF. Opened the tickes with Microsoft but they are just taking forever rather solving the problem.
Tahirah Ahmad 0 Reputation points

2024-07-19T01:21:29.41+00:00

did you figure out an answer? Also ran into this issue

Answer 1

ShaikMaheer-MSFT 38,551 Microsoft Employee Moderator

Hi @Raja ,

Thank you for posting query in Microsoft Q&A Platform.

Since your workflow was working fine but suddenly breaking with error, I suspect this may be driver issue because of some invalid characters in data. This may needs deeper investigation. I encourage you to have a support ticker for this to get deeper investigation and resolution.

Below is the link which explains how to create support ticket in Azure.
https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request

Feel free to share your support ticket details. So that I can internally also escalate for quicker resolution. Thank you.

Raja 86 Reputation points

2021-11-16T13:17:39.313+00:00

Hi @ShaikMaheer-MSFT ,
Thanks for response. I'll certainly create a ticket and come back to you. Apart from what is mentioned in the link as how to create support request, do you want me to take care of anything in specific? Does it cost us anyway?

Thanks,
Raja
ShaikMaheer-MSFT 38,551 Reputation points Microsoft Employee Moderator

2021-11-22T05:11:30.667+00:00

Hi @Raja ,

Depending on your support plan it may cost. Please check below link to know more about Azure Support Plans.
https://azure.microsoft.com/en-in/support/plans/

Share via

ADF is failing for special characters in copy activity

1 answer

Your answer