column delimitter is shifting to new row on new line in quoted text value

Rishabh Patil 161 Reputation points
2022-02-18T15:15:02.107+00:00

Hey Everyone,

I have been trying to read the csv file stored in azure blob using ADF dataflow source

Record in my source file looks like below

175860-image.png

This is my dataset configured below

175874-image.png

And when I am looking at the data preview from source transformation I can see like below

175799-image.png

I am really not getting why its getting into new line.

Any help would be appreciating. thanks

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,670 questions
{count} votes

3 answers

Sort by: Most helpful
  1. Nandan Hegde 36,241 Reputation points MVP Volunteer Moderator
    2022-02-18T16:09:08.947+00:00
    0 comments No comments

  2. Rishabh Patil 161 Reputation points
    2022-02-20T10:28:57.4+00:00

    thanks @Nandan Hegde

    I tried derived column with replace(LongDescription, "\n","") but it didn't help.

    0 comments No comments

  3. ShaikMaheer-MSFT 38,551 Reputation points Microsoft Employee Moderator
    2022-02-21T16:54:30.11+00:00

    Hi @Rishabh Patil ,

    Thank you for posting query in Microsoft Q&A Platform.

    As per my understanding on your ask you trying to understand why your data in data preview is not expected and how to over come this issue. Please correct me if it is wrong.

    In your dataset configurations you have /r,/n or /r/n as row delimiter. Meaning, when ever ADF sees new line it treats as new row and try to take that values from starting column. Hence you are seeing your data preview as above.

    replace(LongDescription, "\n","") this will not help here as this expression is going to evaluate only on "LongDescription" column what you see in data preview.

    For me, it seems your source file data itself is not in proper format. There is not clear delimiter to define new row in file. By seeing data currently in your file its not clear that where exactly the 2nd row should start and where exactly first row is ending.

    I will recommend to have source file data corrected with proper column and row delimiters to make this work.

    Hope this will help. Please let us know if any further queries.

    -----------

    Please consider hitting Accept Answer. Accepted answers helps community as well.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.