Data Factory Pipeline Update

Nathan Carns 196 Reputation points


I have a Data Factory pipeline that has a MariaDB as a source database and an Azure SQL database as destination. I want to be able to add new tables from the source without having to recreate the entire pipeline. How would I do this? Tables and columns are being added to the MariaDB and I'm having to delete and recreate the pipeline every time. I'm also deleting the tables inside the SQL DB. Any way to empty the database from tables and views easily?

Please help with this.


Azure SQL Database
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,491 questions
0 comments No comments
{count} votes

Accepted answer
  1. Ryan Abbey 1,171 Reputation points

    You'll need a pre-step within the pipeline that identifies the tables to pull, you then use a For Each to loop through the list of tables passing a table a time down to your transfer process

0 additional answers

Sort by: Most helpful