Share via

Parsing Error in data flow source.

Amar Agnihotri 921 Reputation points
2023-03-14T10:42:23.5933333+00:00

Hi,

I created a data flow and the source file is json with an array of document.

and its a valid json file as you can see here

User's image

Now i want to flatten this file and only want to extract and save ID column of the data as.csv

User's image

I tried all the three options of in Json settings
User's image

but still i am getting this error

User's image

Can anyone tell me how can i get rid of this error ?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,196 questions

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. AnnuKumari-MSFT 32,161 Reputation points Microsoft Employee
    2023-03-15T20:09:20.4366667+00:00

    Hi @Amar Agnihotri ,

    Thankyou for using Microsoft Q&A platform and thanks for posting your query here.

    As I understand your issue , the source transformation in dataflow is giving error : "Malformed records are detected" even though the json is a valid json.

    I tried to repro your scenario, I tried with a dummy json having similar format like yours and was able to get expected result and didn't experience any error. For seeking better help, I would recommend you to kindly share your complete json content . Here is the video of the dataflow:

    df1

    Hope it helps. Please let me know in case of additional queries, Thanks

  2. Amar Agnihotri 921 Reputation points
    2023-03-20T06:54:11.7266667+00:00

    @AnnuKumari-MSFT

    The issue is resolved now . Actually we need to change the encoding scheme to UTF-8 without BOM in the sink data set of the pipeline which is creating the TempoTeamId.json file . Like here -

    User's image

    I did this and ow dataflow is working fine

    User's image