How to implement a delta connector in a ADF dataflow using Azure .Net SDK (C#)

Question

I have been trying to create a dataflow from AzureSqlDatabase to Delta Format on ADLS Gen2. We have a Sink Type = delta option in the ADF Dataflow UI which enables the direct ingestion instead of an intermediate parquet conversion step / staged copy.

How can we create the dataflow with sink type = delta? Can we implement this delta connector using Azure .net SDK(C#)?

Answer 1

@Divya Rani Pallapothu Thanks for using Microsoft Q&A !!
Yes you can use Delta as a Sink from UI and here is the documentation and a video tutorial for the same. As far as I know you cannot however Delta as a sink in .NET SDK. I see, DataFlowSink class seems using only Datasets while creation. Please provide this as a feedback at Uservoice so that community can upvote and the products team can plan to include it in future releases.

----------

Please do not forget to "Accept the answer" wherever the information provided helps you to help others in the community.

Answer 2

Thanks Saurabh! I could create Delta as a Sink from UI. Also, figured out a way to implement the same via .NET SDK.