Delimeter Issue in the .CSV file

Gurucharan, Satya 41 Reputation points
2022-04-28T02:59:51.79+00:00

Hi All,

I am facing an issue with .csv file format when processing the file with blow data using ADF (Azure Data Factory) the copy pipeline fails with and it is throwing error

|a|,|b|,|c, c 123|,|d|
|a|,|b|,|c, c 123|,|d, 123|
|a|,|b|,|c, c 123|,|d, 123, 1234|

Could any one help or suggest an options

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,085 questions
{count} votes

Accepted answer
  1. fred31330 76 Reputation points
    2022-04-28T20:22:49.433+00:00

    @Gurucharan, Satya , I faced this issue earlier and TBH I solved it by preprocessing it using e.g. regular expressions to replace unwanted characters.

    In your case, replacing pipes with double quotes works:
    197459-image.png

    This is because delimited files can have their fields quoted or not, to delineate a field within a record. Said otherwise, everything between those quotes should not be considered as a field delimiter.


1 additional answer

Sort by: Most helpful
  1. Nandan Hegde 30,951 Reputation points MVP
    2022-04-28T04:33:36.98+00:00

    Based on the error message it seems that in copy activity mapping, you have mapped a column RunDate in sink but that column is not present in your sink .