cell value value changing from apostrophe , to ? during copy activty from on prem to ADLS in ADF

Question

Hello <

I have column with value """ Lane's Sip Lounge"""" in source .txt file , after moving file using copy activity in ADF my column value is changed to """ Lane?s Sip Lounge"""" source dataset column delimiter is PIPE | and sunk data set column delimiter is , comma

Answer 1

Hello AzeemK-2483,

Thanks for the question and using MS Q&A platform.

I suspect this could be because of an Encoding issue. Can you please let me know what is the encoding of the files?

Azure Data factory by default reads the text content in UTF-8. If the source text file has a different encoding and there are chances that this is misinterpreted.

Are these generated programmatically ? If yes, you could check in which encoding are these files written.

With that information, you could provide the source file's encoding in the Azure Data Factory and see whether this resolves the issue.

The destination/sink can be in UTF-8 or any other encoding by choice.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators