How to copy a file from azure data lake storage gen 1 to a network drive using python

Question

Our azure databricks cluster is restricted with saving spark data frame to a network drive directly, hence we have to save it as a csv file to our azure data lake storage gen1, then copy it to a network drive, we have successfully saved the spark data frame as a csv file to our data lake storage gen 1, however, i have not found a solution as how to copy the file from the lake to a specific net work drive using python script (the network drive will be varied as a parameter based on user who runs the command), anyone can help at all? Thanks.

Answer 1

Hello @azurefan ,

Welcome to the Microsoft Q&A platform.

You may checkout the below articles to download a file from Azure Data Lake Storage Gen1 to your network drive.

• Service-to-service authentication with Azure Data Lake Storage Gen1 using Python
• Account management operations on Data Lake Storage Gen1 using Python
• Data operations on Data Lake Storage Gen1 using Python

And also, you may checkout the below threads which addressing similar issue:

Copy File/Folders in Azure Data Lake Gen1

Extracting Data from Azure Data Lake Store Using Python: Part 1 (The Extracting Part)

Extracting Data from Azure Data Lake Store Using Python: Part 2 (The Design Part)

Hope this helps. Do let us know if you any further queries.

------------

Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.