ADF pipeline failing to read CSV file if a column values contains comma delimeter anlong with double quotes.

Question

175228-sample-data.txtI have a CSV file which is comma (,) separated and in a column value (Column D) it contains comma delimiter(,) along with Double doutes as below.

This was more "confirming" for me than "enlightening." I did however, pick up a new priority with regard to sharing my experience.

i'm using Copy activity to load this file and below are the parameters i have set while creating a dataset.

Now when i try Copy activity to read this files it says "Message=Error found when processing 'Csv/Tsv Format Text' source 'tmp.csv' with row number 4186: found more columns than expected column count 15".

How can i fix this issue.

Answer 1

Hi @Mishra,Muni ,
Welcome to Microsoft Q&A platform and posting your query.
My understanding about your query is that you have a CSV file where one of the columns is having data which carries double quotes as well as comma. You want to store that as a single column value in the sink . Seeing the error , it looks like the sink dataset is having different number of columns pre-defined. It would be better if you could share more details about the sink dataset.

• First and foremost , the attached file is a .txt file but you want to process it as csv data . Please save the file manually as .csv and upload in the storage account.
• Select 'Double quotes' in the escape character field so that the whole string will be considered as one column data.

Please refer to the attached screenshot:

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you.
  Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators