Azure Databrick dbfs file uploaded and folder created not listing in notebook and vice versa.

Question

Azure Databrick dbfs file uploaded and folder created not listing in notebook and vice versa.

Bibin Kunjumon 0

I have created multiple folders in location dbfs:/FileStore/demo_store using Browse DBFS.

These folders are not listed when accessing from notebook using %ls.

Dont think,its due to syntax.Nop.

I was working with folder and uploaded file in morning.Now when i created new folder its not working. And also the folders created by notebook using mkdir command listed in notebook but not displayed in explorer/browse

User's image

dbfs:/FileStore/demo_store/built_model/trained_vggface.h5

In code(notebook) i am using :
This not works - %ls /FileStore/demo_store/trained_models/

This works. - %ls /FileStore/demo_store/

the folder created in morning and files uploaded in morning are still accessible from notebook.But now when i upload in explorer its not showing in notebook, and saved in notebook not shown in explorer.

VasimTamboli 5,215 Reputation points

2023-06-04T11:53:59.57+00:00

Refresh the notebook and explorer/browse: In the notebook, you can try refreshing the page or running %refresh to update the file and folder listing. In the explorer or browse section, click the refresh button or try navigating to a different folder and then back to the desired location to trigger a refresh.

Clear browser cache: Clear the cache and cookies in your browser to ensure that you're viewing the latest state of the Databricks workspace. Sometimes, cached data can cause inconsistencies in displaying newly created or uploaded files and folders.

Check for synchronization delays: Sometimes, there may be synchronization delays between the backend storage and the Databricks UI components. It can take a few seconds or minutes for changes made in one location to be reflected in the other. If you've recently created or uploaded files/folders, give it some time and try refreshing the notebook and explorer/browse again.

Verify folder path: Double-check the folder path you're using in the %ls command to ensure it matches the actual location of the files and folders in the DBFS. Check for any typos or inconsistencies in the path that could be causing the issue.

Restart the cluster: In some cases, restarting the cluster in Azure Databricks can help resolve caching or synchronization issues. Try restarting the cluster associated with your notebook to see if it resolves the problem. Be aware that restarting the cluster may cause a temporary disruption in your running jobs or computations.

If the issue persists even after trying these steps, it would be helpful to reach out to the Azure Databricks support team for further assistance. They can examine the specific details of your workspace and provide guidance based on your configuration.

Additionally, ensure that you have the necessary permissions and access rights to view and interact with files and folders in the DBFS. It's possible that the issue could be related to permissions or access restrictions set up in the Databricks workspace.

I hope these suggestions help resolve the inconsistency in displaying folders and files between the notebook and explorer/browse in Azure Databricks.
QuantumCache 20,366 Reputation points Moderator

2023-06-20T01:17:01.21+00:00

Hello @Bibin Kunjumon, Just checking if we are still connected on this discussion? Please let us know and we are happy to help you further.

Your answer

QuantumCache 20,366 Reputation points Moderator

2023-06-20T01:17:01.21+00:00

Hello @Bibin Kunjumon, Just checking if we are still connected on this discussion? Please let us know and we are happy to help you further.

Share via

Azure Databrick dbfs file uploaded and folder created not listing in notebook and vice versa.

Your answer