Error in python, why?

NishimuraChinatsu-9854 756 Reputation points
2022-09-05T08:20:13.537+00:00

Is it possible to use os.listdir()?

I have additional questions, does the same problem occur with AzureFunction?

Is there a way to load excel files in a list?
I am then looking to process each excel file in the folder.

237763-tempsnip.png

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
4,303 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,401 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. MartinJaffer-MSFT 26,031 Reputation points
    2022-09-08T01:53:25.757+00:00

    Hello @NishimuraChinatsu-9854 ,
    Thanks for the question and using MS Q&A platform.

    As I understand, the ask is whether you can use the python os.listdir() in Synapse notebook, to navigate Azure Datalake Gen2. Also whether similar happens with Azure Functions.

    Sorry I took so long, but I needed to make some corrections. I thought os.listdir could be used in conjunction with mounting, but this doesn't seem to be the case. I must have gotten confused with Databricks behavior. This excerpt from mounting api documentation doesn't help either. I'll get with my colleagues for clarification.

    After the storage is in place, you can use the local file API to access data as if it's stored in the local file system

    os.listdir cannot be used with abfss. This is because they are different protocols. os.listdir expects native paths like C:\mydir/mysubfolder/myfile . These can be posix or windows type file paths, (I think?).
    However, abfss:// is a totally different protocol. If you look at it, it seems more similar to a URL, than a filepath.
    Other operations CAN use the abfss, such as dataframe, pandas, and some spark.

    Alternatively, there are other ways to navigate the storage.

    You can use the mssparkutils to list the contents of a mount.
    fsspec can do ls

    Please do let me if you have any queries.

    Thanks
    Martin

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
      • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
    0 comments