JSON sink in dataflow generates empty file whereas data preview shows data

Question

JSON sink in dataflow generates empty file whereas data preview shows data

Meher, Shweta 1

We have built a Dataflow pipeline which is exporting data in json format which has array struct .I am able to preview the data in preview and able to export it in CSV from preview window.But when I run the pipeline ,it completes with success generating empty json file .All the datatype is verified and validated.

Smaran Thoomu 25,005 Reputation points Microsoft External Staff Moderator

2024-08-01T06:23:25.8766667+00:00

@Meher, Shweta Just checking in to see if the below answer provided by @hossein jalilian helped.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

1 answer

Your answer

Smaran Thoomu 25,005 Reputation points Microsoft External Staff Moderator

2024-08-01T06:23:25.8766667+00:00

@Meher, Shweta Just checking in to see if the below answer provided by @hossein jalilian helped.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 1

hossein jalilian 11,055 Volunteer Moderator

Thanks for posting your question in the Microsoft Q&A forum.

Ensure that your source files contain valid data and remove any empty files and double-check that all data types are correctly set and match your source data structure.

Experiment with different partitioning strategies to see if it improves the output, insert additional sink steps throughout your data flow to identify where the data might be getting lost and try running the pipeline with a small subset of your data to isolate the issue.

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful

Smaran Thoomu 25,005 Reputation points Microsoft External Staff Moderator

2024-08-02T16:44:50.37+00:00

@Meher, Shweta Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

JSON sink in dataflow generates empty file whereas data preview shows data

1 answer

Your answer