issue converting parquet format to csv in ADF

Question

Having the issue converting parquet data type to csv in ADF.
I have a pipeline to copy from one blob to another replacing the format of the file from parquet to csv.
Here is a sample of the data:

header1 header2 header3 header4 header5
1107672220 474 Straight 5/8" N/A Y 12/31/9999
1107679364 471 5/8" Y 12/31/9999
1103448910 67 Snacks, Coffee, and Pet Y 6/22/2008

For cleaner view the same but on the screenshot:

The output csv is then formatted incorrectly. It may be broken differently depending on the settings in a sink data set – escape character or quote character. have tried several options but none of them work for me. Here are a few of them:

1. Data for some rows in 3rd column are combined
   
2. In this one data are split by delimiters where it shouldn’t be. Extra columns were created
   
3. I used different settings, but the output is similar as in 1)

I tried different set of configurations but none of them works.
There are similar threads here none of them clearly indicates the solution.

https://stackoverflow.com/questions/61593352/azure-data-factory-escape-character-and-quote-issue-copy-activity
https://learn.microsoft.com/en-us/answers/questions/115904/azure-data-factory-adf-escape-character-issue-when.html

Answer 1

Hi @braxx ,

Please try with the following settings. Hope this helps! :)