How to migrate datasets in ADF from Gen 1 to Gen2

Question

How to migrate datasets in ADF from Gen 1 to Gen2

Erika Gupta 0

Hi ,

I need to migrate datasets in data factory to gen 2 , which are currently pointing to Gen1 datalake.

We have around 500 datasets pointing to Gen1 Datalake.

How can we make them point to Gen2 datalake?

1 answer

Your answer

Answer 1

Tech-Hyd-1989 5,816

Hi Erika Gupta

Good day!

Azure Data Factory offers a scale-out, managed data movement solution. The article below shows you how to use the Data Factory copy data tool to copy data from Azure Data Lake Storage Gen1 into Azure Data Lake Storage Gen2. You can follow similar steps to copy data from other types of data stores.

https://learn.microsoft.com/en-us/azure/data-factory/load-azure-data-lake-storage-gen2-from-gen1

Because of the scale-out architecture of Data Factory, it can ingest data at a high throughput. For more information, you can see Copy activity performance.

Please accept as answer and upvote if the above information is helpful for the benefit of the community.

Erika Gupta 0 Reputation points

2023-03-01T05:56:21.7533333+00:00

Hi Subroto,

Thanks for this information.

The challenge I am facing now is not migrating the data but the Source /Sink datasets which are being used in pipelines.

Thanks

Erika
PRADEEPCHEEKATLA 90,616 Reputation points

2023-03-02T07:51:58.6633333+00:00

@Anonymous
What do you mean by not migrating the data? Are you experiencing any error message?

Please do share the steps which you have tried along with the error message if any?
PRADEEPCHEEKATLA 90,616 Reputation points

2023-03-08T07:46:14.2566667+00:00

@Erika Gupta

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Erika Gupta 0 Reputation points

2023-03-09T06:40:19.3866667+00:00

Migrating data from Gen1 to Gen2 is easily accomplished.

We are facing issue in migrating the datasets (Source/Sink)which are pointing to Gen1.
PRADEEPCHEEKATLA 90,616 Reputation points

2023-03-13T04:27:31.8333333+00:00

@Erika Gupta Can you please do share the steps which you have tried along with the error message if any?

Share via

How to migrate datasets in ADF from Gen 1 to Gen2

1 answer

Your answer