This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 8%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEF%3C/text%3E%3C/svg%3E)
I have an Azure Data Factory that copies Azure SQL source to Azure blob storage delimited text sink with DataFlow.
The data contains translations for different languages. The data can contain double quote values. I've set the Quote and Escape character to double quote (").
Some values are escaped correctly:
But other values are not escaped correctly:
The strange part is that when exporting the preview data to CSV all data is escaped correctly?
How can I create valid CSV files from my source?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 9%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
@Eric Fitskie Welcome to Microsoft Q&A forum and thanks for reaching out here.
Yes, this seems strange. To help reproduce/replicate the issue on our end, and escalate to respective product owners, will it be possible for you to provide a mock file similar to that of you source?
Thank you
Hi @KranthiPakala-MSFT , Thank you for your response.
If you use the query below as Azure SQL source you can reproduce the issue:
select 1 as id, N'"quoted" value' as value
union all
select 2 as id, N'Система загальних втрат' as value
union all
select 3 id, N'API: CE + 6 % vådbremseadditiv' as value
union all
select 4 as id, N'API: CE + 6 % присадки "мокрого" гальмування' as value
Kind Regards
@Eric Fitskie Thank you for details, I'm trying to reproduce the issue on my end and will update you with my findings shortly.
@Eric Fitskie Is your requirement just to escape the double quote characters in your values or to escape characters and enclose in double double quotes?
If your requirement is to just escape the double quote characters, then you can configure the dataset as below :
No quote characterDouble quote (")But if your requirement is to escape the double quote characters and enclose the whole column value in a double quote character, then I recommend filing a support ticket for deeper investigation. Please do file a support ticket and share the Support ticket number with us here for tracking it internally. In case if you don't have a support plan, please let me know here.
FYI - In parallel, I'm also reaching out to internal teams to see if they have any inputs with regards to this issue. I'm suspecting it could be a bug but have to hear back from the product team.
I even tested the same with Copy activity and it is working as expected so suspecting this issue specific to dataflow. I'll keep you posted as soon as I hear back from the product team.
Thank you
Hi @KranthiPakala-MSFT , Thank you for your investigation and options.
My requirement is to produce CSV files that can be read by most of the common CSV parsers around.
The data can contain almost any character, including separator value and different kind of quotes.
Setting the Quote character = No quote character does not work for the values that contain the separator value.
The Quote All option does produce valid CSV files as far as i can tell, but this does increase the file size and increments processing time. This could be used as work around, but having this fixed in the original situation is what i prefer.
Unfortunately I do not have an support plan yet (working on that). So if you could help me with this that will be appreciated.
@Eric Fitskie Thanks for the feedback. Please use the workaround to mitigate the problem. And If you don't have a support plan, could you please send an email to AzCommunity[at]Microsoft[dot]com with the below details, so that we can work with you in creating a one-time-free support ticket for you to further investigate the original issue.
Email subject: <Attn - Kranthi : Microsoft Q&A Thread title>
Thread URL: <Microsoft Q&A Thread>
Subscription ID: <your subscription id>
Looking forward to your reply.
Thank you
@Eric Fitskie Following up to check if you have got a chance to send the email with details requested. Kindly confirm here once the email is sent.
Thank you
Hi @KranthiPakala-MSFT , Sorry the comments were hidden and i missed them. I've sent you the mail
Kind Regards,
@Eric Fitskie Could you please try to use Quote all feature under sink transformation settings and see if that helps meet your requirements.
This is how the output will look like:
Let me know if this helps.
Thank you!