writing to sink

arkiboys 9,666 Reputation points
2022-03-17T08:24:48.907+00:00

Hello,
in dataflow, I am able to write to sink in gen2.
linked service and dataset connected to gen2

1-
To check if all data is written to this storage folder, i.e. delivery, I have a dataset which points to this folder.
When I preview it, does it show all rows or some of them? it looks like it shows some of them.?

2-
Why is it that in the storage I only see .parquet files in the delivery folder and there are no subfolders, i.e. for dataset that I am passing to the dataset connection
I have set the clear folder to true thinking that if the same parameter exists as subfolder under delivery folder, then it is deleted first before it gets written to it.
But this does not seem to be happening.

184052-image.png
183988-image.png
184043-image.png

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,751 questions
0 comments No comments
{count} votes

Accepted answer
  1. ShaikMaheer-MSFT 38,201 Reputation points Microsoft Employee
    2022-03-17T17:08:48.213+00:00

    Hi @arkiboys ,

    Thank you for posting query in Microsoft Q&A Platform.

    As per my understanding asks are, Is data preview shows only partial data? and What Clear folder option will do. Please correct me if my understanding is wrong.

    Yes, Data preview option will show only partial data not all the data & Clear the folder Determines whether or not the destination folder gets cleared before the data is written.

    Hope this helps. Please let us know if anything further queries. Thank you.

    ------------

    Please consider hitting Accept Answer. Accepted answers helps community as well.

    0 comments No comments

0 additional answers

Sort by: Most helpful