Share via

listing files from adls gen2 from databricks

Hari, Jayashree (Cognizant) 1 Reputation point
2022-03-22T19:38:17.86+00:00

How to list files from ADLS gen2 of "/path/.filetype" in databricks?
dbutils.fs/ls("path/.parquet") is considered as actual filename

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,426 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,080 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. PRADEEPCHEEKATLA-MSFT 85,586 Reputation points Microsoft Employee
    2022-03-23T04:26:03.45+00:00

    Hello @Hari, Jayashree (Cognizant) ,

    Thanks for the question and using MS Q&A platform.

    Azure Databricks dbutils doesn't support all UNIX shell functions and syntax, so that's probably the issue you ran into.

    Note: %sh reads from the local filesystem by default. To access root or mounted paths in root with %sh, preface the path with /dbfs/.

    Try using a shell cell with %sh to get the list files based on the file type as shown below:

    185897-image.png

    For more details, refer Databricks File System (DBFS).

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators