how to mounting azure storage ADLSgen2 all continers in azure databricks

Question

Hi,

How to mount azure storage adls gen2 all containers from azure databricks using service principle. Please advice step by step. There are codes availale for single continer mounting. I need to mount all the containers in adls gen2 storage account.

Thanks

Answer 1

Hello Ram Babu,

Welcome to the Microsoft Q&A forum.

To mount multiple containers in Azure Databricks, you can create a list of mount points and iterate through them to mount each container.

Please see the below code for your reference.

I have tested the code and am able to mount multiple containers.

configs = {"fs.azure.account.auth.type": "OAuth",
          "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
          "fs.azure.account.oauth2.client.id": "<application-id>",
          "fs.azure.account.oauth2.client.secret": dbutils.secrets.get(scope="<scope-name>",key="<service-credential-key-name>"),
          "fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/<directory-id>/oauth2/token"}

# Define a list of containers and mount points

containers = [
  {"source": "abfss://container1@yourstorageaccount.dfs.core.windows.net/", "mount_point": "/mnt/excel/container1/"},
  {"source": "abfss://container2@yourstorageaccount.dfs.core.windows.net/", "mount_point": "/mnt/excel/container2/"},
  # Add more containers as needed
]

# Iterate through the list and mount each container
for container in containers:
  dbutils.fs.mount(
    source=container["source"],
    mount_point=container["mount_point"],
    extra_configs=configs
  )

I hope this helps. Please let me know if you have any further questions.