Cannot mount Azure VM(windows server) file share on Azure Databricks instance.

Question

Hi,

I wanted to mount windows file share(from Windows server VM in Azure) on a databricks instance. To test it I followed these steps.

1. I created a Windows VM and a Linux VM in azure.
2. I created VNet-peering between these two VMs.
3. After creating the VNet-peering, windows VM was able to ping the private IP address of the Linux VM and vice-versa.
4. I used the command sudo mount -t cifs -o username=XXXX //x.x.x.x/Downloads ./local
5. The mount worked. I could list the files in the remote directory.
6. Similarly, I created a databricks workspace in the same vNet which is being peered with the windows VM virtual network.
7. After creating a cluster, I tried to mount the windows file share from the cluster terminal. The mount is not working.
8. To check the communication between the databricks instance and windows vm, I tried pinging them from each other.
9. Now, ping from databricks instance is reaching the private ip of the windows vm. but windows vm cannot ping databricks private ip. windows vm can ping default gateway of databricks instance. we need some config so that pings from windows vm to databricks private ip.
10. This is the problem, I believe if the ping works, we will be able to mount the file share.

Here are some screenshots:

SS from databricks terminal. Private IP: 10.2.3.4 subnet CIDR: 25

Databricks pinging WindowsVM1

Default gateway of databricks: 10.2.2.4

WindowsVM1 receiving pings from databricks default gateway.

WindowsVM1 sending failed pings to databricks private ip 10.2.3.4

WindowsVM1 sending successful pings to databricks default gateway 10.2.2.4

Answer 1

Hello @Sai Suresh (Quadrant Resource) ,

Welcome to the Microsoft Q&A platform.

Unfortunately, Azure Databricks doesn't support mounting Azure VM(windows server) file share.

Azure Databricks supported data sources: https://learn.microsoft.com/en-us/azure/databricks/data/data-sources/

I would suggest you to provide feedback on the same:

https://feedback.azure.com/forums/909463-azure-databricks

All of the feedback you share in these forums will be monitored and reviewed by the Microsoft engineering teams responsible for building Azure.

You may checkout SO thread which addressing similar issue.

Hope this helps. Do let us know if you any further queries.

------------

Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.