Delta format in Azure Data Factory

Question

Delta format in Azure Data Factory

Shobhit Kumar Agrawal 1 Microsoft Employee

Currently we are using DataFactory to pull the data using Parquet format, we do have scenario where source team is providing delta format and one single file with parquet format.

we have seen the article where we do have option to use Dataflow to consume data using delta format but seems it is in preview, i want to know what is the plan to make it production so that we can use this feature for our production environment.

https://learn.microsoft.com/en-us/azure/data-factory/format-delta

1 answer

Your answer

Answer 1

KranthiPakala-MSFT 46,642 Microsoft Employee Moderator

Hi @Shobhit Kumar Agrawal ,

Welcome to Microsoft Q&A and thanks for reaching out.

Delta format in Azure Data Factory is available in GA - Please refer to this article : ADF Data Flow Updates: CDM and Delta Lake Connectors Generally Available

Known limitation : When writing to a delta sink, there is a known limitation where the numbers of rows written won't be return in the monitoring output.

Hope this info helps.

----------

Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.

KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2021-05-28T06:38:32.767+00:00

Hi @Shobhit Kumar Agrawal ,

Just checking in to see if the above suggestion was helpful. If this answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2021-06-01T19:46:01.527+00:00

Hi @Shobhit Kumar Agrawal ,

We still have not heard back from you. Just wanted to check if the above information was helpful? If it answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread.

And, if you have any further query do let us know.

Share via

Delta format in Azure Data Factory

1 answer

Your answer