Share via

Is Azure Durable function are efficent/suitable choice to dump around 1 billion records from cloud based Data warehouse to RDBMS?.

Cloud native user 1 Reputation point
2021-05-20T13:43:56.187+00:00

We want to create once a day activity to copy Data from Cloud based Dataware house to Azure based RDBMS.
Record count we are expecting daily is around 1 billion record size is small. Azure durable functions will be a good choice as compared to a microservice ? Or there are other suitable cloud native solutions ?

Azure Cloud Services
Azure Cloud Services
An Azure platform as a service offer that is used to deploy web and cloud applications.
677 questions
Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
4,679 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. JayaC-MSFT 5,526 Reputation points
    2021-05-25T12:52:27.81+00:00

    Hello @Cloud native user , In this scenario, you should consider Azure Data factory

    Azure Data Factory is the platform that solves such data scenarios. It is the cloud-based ETL and data integration service that allows you to create data-driven workflows for orchestrating data movement and transforming data at scale. Using Azure Data Factory, you can create and schedule data-driven workflows (called pipelines) that can ingest data from disparate data stores. You can build complex ETL processes that transform data visually with data flows or by using compute services such as Azure HDInsight Hadoop, Azure Databricks, and Azure SQL Database.

    https://azure.microsoft.com/en-in/services/data-factory/#features

    https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-sql-database

    Please let me know if this helps.

    0 comments