Share via

Working directory in Databricks Repos - Path when creating temporary files on the databricks driver

Jonathan Athot 1 Reputation point
Sep 29, 2022, 8:07 PM

Hi,

In Databricks runtime 10.4 LTS, when we are not in a repository, in R we can call the function getwd() and it return the working directory. For example, it can return "/tmp/Rserv/conn1491". The same can be done with Python with the function os.getcwd().

However, when the code are in a Repos then the function will return where the notebook is in the Repos. It’s fine if you need to call other notebook with relative path. However, if I create a file (ex : csv, pdf, txt, …) and want to copy it elsewhere then I need the full path of this file that will be created in the temporary folder associate with the notebook session (when not in a Repos it was for example "/tmp/Rserv/conn1491").

How can we retrieve this path (in R and in Python) if the function now don’t give us this information ?

Regards,

Jonathan

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,331 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. HimanshuSinha-msft 19,476 Reputation points Microsoft Employee
    Oct 1, 2022, 12:46 AM

    Hello @Jonathan Athot ,
    Thanks for the question and using MS Q&A platform.
    As it appears to me you want the access some csv/txt/pdf file in the notebook . I suggest you to please DBFS files system exposes . Please read about that here .
    https://docs.databricks.com/files/index.html

    246588-image.png

    Please do let me if you have any queries.
    Thanks
    Himanshu


    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
      • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.