Handle comma(,) in column value in comma separated delimited text file

Question

I have comma separated delimited file as input and using ADF copy activity to copy it to a .CSV file
Now if a field has a comma(,) in its value, obviously it treats the rest of the field value as the next column and the copy aborts since it is having one extra column than expected.

How to escape the comma(,) in between the field value?

Answer 1

Hello @Utsav Chanda ,

Thanks for the question and using Microsoft Q&A.

One thing I observed, when I had used MS Excel to create an csv through Excel. I saw that when the comma (,) was present in the FieldValue. There were quotes ("") between the text that contained comma (,).

The ADF was able to process the above CSV file.

If the field values that contained comma(,) are not quoted (""), then I encounter same error message mentioned above.

I can think of 2 option at our end:

**Option 1 : **

you could configure the source that is generating the comma separated files to quote either all field values or the field values that contain comma(,).

TestText,"V,S",29  

OR

"TestText","V,S","29"  

This is to differentiate between comma in column separation and comma in field value.

Alternatively, you could check whether you could change the delimiting character at the source.

Option 2 :

If the above is not viable option for you, you could choose to skip rows that are incompatible (i.e. field values with comma) with the below configuration under the setting section.

This is skips all the rows that are incompatible and you would also get a log stating which row in the file has been skipped. The copy activity is carried out sucessfully for the rest of the rows.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators