question

AlphonseOkossiFT-4385 avatar image
0 Votes"
AlphonseOkossiFT-4385 asked AlphonseOkossiFT-4385 edited

do you have python code to specifically load load data from on-premise MySQL database into azure data lake storage?

do you have python code to specifically load load data from on-premise MySQL database into azure data lake storage?

  • I am not finding a specific example for MySQL in Azure Data Factory

  • preferably Python example

azure-data-lake-storage
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
0 Votes"
PRADEEPCHEEKATLA-MSFT answered AlphonseOkossiFT-4385 edited

Hello @AlphonseOkossiFT-4385,

As per my knowledge, I haven't seen anyone using python code to load data from on-premise MYSQL database into Azure Data Lake Storage.

Pushing data from your on-premise mysql database into the cloud can easily be orchestrated with Azure Data Factory. Azure Data Factory is Azure’s cloud-based data integration service that allows you to orchestrate and automate data movement and transformations.

If you are using Microsoft Azure Cloud, using ADF is the way to go:

  • ADF takes care of all needed drivers to integrate with Oracle, MySQL or SQL Server; together accounting for +90% of all on-premise databases.

  • ADF has a solid way to access data, securely over your firewall through a gateway.

  • If your data store is located inside an on-premises network, an Azure virtual network, or Amazon Virtual Private Cloud, you need to configure a self-hosted integration runtime to connect to it.

81095-image.png

For more details, refer Copy data from MySQL using Azure Data Factory and Moving Data from an On-Prem Database Using Azure Data Factory.

Hope this helps. Do let us know if you any further queries.


Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.



image.png (65.3 KiB)
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @AlphonseOkossiFT-4385,

Just checking in to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

0 Votes 0 ·

Hello @AlphonseOkossiFT-4385,
Following up to see if the above suggestion was helpful. And, if you have any further query do let us know.
Take care & stay safe!

0 Votes 0 ·

Thank you for your answer. Very helpful.

Followup question: we have VPN to connect to the on-premise MySQL database, how would you modify the above architecture to account for that?

Also is there a way to use a VM on Azure to do the processing of data pull from the on-premise MySQL rather than using self-hosted integration runtime?

0 Votes 0 ·