How to mount dataset or datalake with read/write permission in Notebook of Azure Machine Learning Studio?

Question

How to mount dataset or datalake with read/write permission in Notebook of Azure Machine Learning Studio?

I've successfully mounted a dataset with read permission to notebook of my machine learning studio account.

But when I try to write back to the datalake/dataset, it throughs "[Errno 30] Read-only file system".

How can I mount the dataset with write permission?


from azureml.core import Workspace, Dataset, Datastore

import tempfile

from azureml.dataprep.fuse.dprepfuse import MountOptions

mount_options = MountOptions(default_permission=0o777)

mounted_path = tempfile.mkdtemp()  

subscription_id = '**********************'

resource_group = '**********************'

workspace_name = '**********************'

  

workspace = Workspace(subscription_id, resource_group, workspace_name)

  

datastore = Datastore.get(workspace, "my_datalake")

dataset = Dataset.File.from_files(path=(datastore, 'ml/folder1/folder2/'))

mount_context = dataset.mount(mounted_path, mount_options=mount_options)

mount_context.start()

print(mounted_path)

import os

os.mkdir(f"{mounted_path}/newfolder", )

OSError                                   Traceback (most recent call last)

Cell In[2], line 22

 18 print(mounted_path)

 20 import os

---> 22 os.mkdir(f"{mounted_path}/john", )

 24 # refer to this

 25 # https://learn.microsoft.com/en-us/answers/questions/884340/read-only-file-system-error-in-azureml-studio

 26 # https://learn.microsoft.com/en-us/answers/questions/67126/failure-exception-oserror-(errno-30)-read-only-fil

 27 

 28 # az ml datastore attach-blob --account-name cgrpdatalakepoc --container-name filesystempoc --name ds_datalake_poc

OSError: [Errno 30] Read-only file system: '/tmp/tmpfyqld99p/newfolder'

ame: azureml_py38

channels:

  - conda-forge

  - anaconda

  - defaults

dependencies:

  -...

romungi-MSFT 30,291 Microsoft Employee

Milad Resketi Try using the relative current path instead of a temp path since I don't think the mount options are taking any effect here since the you are actually using a kernel with the notebook instead of actually using the path on the compute instance.

   with dataset.mount() as mount_context:
       # list top level mounted files and folders in the dataset
       os.listdir(mount_context.mount_point)

   # You can also use the start and stop methods
   mount_context = dataset.mount()
   mount_context.start()  # this will mount the file streams
   mount_context.stop()  # this will unmount the file streams

romungi-MSFT 30,291 Reputation points Microsoft Employee

2023-03-21T17:18:59.0766667+00:00

Milad Resketi Did the above suggestion help to mount the dataset? Thanks!!

romungi-MSFT 30,291 Reputation points Microsoft Employee

2023-03-17T11:25:40.1233333+00:00

Milad Resketi Try using the relative current path instead of a temp path since I don't think the mount options are taking any effect here since the you are actually using a kernel with the notebook instead of actually using the path on the compute instance.

with dataset.mount() as mount_context: # list top level mounted files and folders in the dataset os.listdir(mount_context.mount_point) # You can also use the start and stop methods mount_context = dataset.mount() mount_context.start() # this will mount the file streams mount_context.stop() # this will unmount the file streams
romungi-MSFT 30,291 Reputation points Microsoft Employee

2023-03-21T17:18:59.0766667+00:00

Milad Resketi Did the above suggestion help to mount the dataset? Thanks!!

How to mount dataset or datalake with read/write permission in Notebook of Azure Machine Learning Studio?

Activity