How to copy a file from azure data lake storage gen 1 to a network drive using python

azurefan 1 Reputation point

Our azure databricks cluster is restricted with saving spark data frame to a network drive directly, hence we have to save it as a csv file to our azure data lake storage gen1, then copy it to a network drive, we have successfully saved the spark data frame as a csv file to our data lake storage gen 1, however, i have not found a solution as how to copy the file from the lake to a specific net work drive using python script (the network drive will be varied as a parameter based on user who runs the command), anyone can help at all? Thanks.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
993 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA-MSFT 59,336 Reputation points Microsoft Employee

    Hello @azurefan ,

    Welcome to the Microsoft Q&A platform.

    You may checkout the below articles to download a file from Azure Data Lake Storage Gen1 to your network drive.

    And also, you may checkout the below threads which addressing similar issue:

    Copy File/Folders in Azure Data Lake Gen1

    Extracting Data from Azure Data Lake Store Using Python: Part 1 (The Extracting Part)

    Extracting Data from Azure Data Lake Store Using Python: Part 2 (The Design Part)

    Hope this helps. Do let us know if you any further queries.


    Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.