Merging files from two different containers or storage accounts.

asked 2021-03-30T21:06:28.847+00:00
Ade Kunah 1 Reputation point

I have some files in two different containers or storage accounts and would like to make these files available in a single container or single storage account, can this be archieved using ADF?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,261 questions
{count} votes

2 answers

Sort by: Most helpful
  1. answered 2021-03-31T05:45:03.547+00:00
    Vaibhav Chaudhari 37,746 Reputation points

    One option is to have 2 copy activities which will copy these 2 files in a single folder called staging.

    Then one more Copy activity, which will point to staging folder and merge 2 files and save it to some target folder. In Copy activity property, you need to set copy behavior to merge as shown in below -

    https://stackoverflow.com/questions/56550727/azure-data-factory-how-to-merge-all-files-of-a-folder-into-one-file


    Please don't forget to Accept Answer and Up-vote if the response helped -- Vaibhav

    No comments

  2. answered 2021-04-02T05:00:17.413+00:00
    Kiran-MSFT 686 Reputation points

    You can use dataflow to copy from multiple sources into a single storage account without staging the data.

    No comments