Mount blob storage when Run container applications on Azure Batch

Woody-Azure 1 Reputation point
2021-09-07T07:54:44.7+00:00

Hi,

I was trying to mount blob storage to my container image when I run container application on Azure Batch with python. I used the sample code based on https://learn.microsoft.com/en-us/azure/batch/batch-docker-container-workloads.

I added some codes to mount azure blob. I could find some references from _models.py in azure batch models and https://github.com/microsoft/CameraTraps/blob/e5b7b55604b260895e2796d34e9e14db24920ca0/api/batch_processing/api_support/create_batch_pool.ipynb

Here is my code when I tried to create batch pool. I think I followed all the material that I could find, but it still not work with MountConfigurationError as below image.
129778-image.png 

image_ref_to_use = batch.models.ImageReference(  
        publisher='microsoft-azure-batch',  
        offer='ubuntu-server-container',  
        sku='16-04-lts',  
        version='latest'  
    )  
  
    # Specify a container registry  
    # We got the credentials from config.py  
    containerRegistry = batchmodels.ContainerRegistry(  
        user_name=config.__REGISTRY_USER_NAME,  
        password=config.__REGISTRY_PASSWORD,  
        registry_server=config.__REGISTRY_SERVER  
    )  
  
    # The instance will pull the images defined here  
    container_conf = batchmodels.ContainerConfiguration(  
        container_image_names=[config.__CONTAINER_IMAGE],  
        container_registries=[containerRegistry]  
    )  
      
  
    azure_blobfile = batchmodels.AzureBlobFileSystemConfiguration(  
        account_name=  "batchstorage",  
        container_name="test",  
        account_key=config.__STORAGE_ACCOUNT_KEY,  
        blobfuse_options="-o attr_timeout=240 -o entry_timeout=240 -o negative_timeout=120 -o allow_other",  
        relative_mount_path="tmp"  
    )  
    mount_conf = batchmodels.MountConfiguration(  
        azure_blob_file_system_configuration = azure_blobfile  
    )     
      
    new_pool = batchmodels.PoolAddParameter(  
        id=pool_id,  
        virtual_machine_configuration=batchmodels.VirtualMachineConfiguration(  
            image_reference=image_ref_to_use,  
            container_configuration=container_conf,  
            node_agent_sku_id='batch.node.ubuntu 16.04'),  

        vm_size='STANDARD_D4_v3',  
        target_dedicated_nodes=config.__DEDICATED_POOL_NODE_COUNT,  
        target_low_priority_nodes=config.__LOW_PRIORITY_POOL_NODE_COUNT,  
        mount_configuration=[mount_conf]  
    )

If you help me how to mount blob storage, it would be great help.

Thanks!

Azure Container Registry
Azure Container Registry
An Azure service that provides a registry of Docker and Open Container Initiative images.
389 questions
Azure Batch
Azure Batch
An Azure service that provides cloud-scale job scheduling and compute management.
302 questions

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. prmanhas-MSFT 17,886 Reputation points Microsoft Employee
    2021-09-29T14:45:33.187+00:00

    We are working on this issue with @Woody-Azure and will keep thread posted once the issue is resolved

    0 comments
  2. prmanhas-MSFT 17,886 Reputation points Microsoft Employee
    2021-10-12T05:59:21.483+00:00

    @Woody-Azure Thank you for your cooperation on the matter!!!

    As per internal investigation after Checking the logs it looks like there was competition for a lock on the apt package manager preventing package download. One possible resolution was creating another node in the pool to see if the problem persists? We checked the Microsoft repo to ensure the package exists and it does, so it's likely just an installation issue. Also Ubuntu 16.04 is end of life (EOL) meaning it is no longer supported by Batch. We did suggested upgrading to Ubuntu 20.04 when possible

    Hope it helps!!!

    Please "Accept as Answer" if it helped so it can help others in community looking for help on similar topics.

    0 comments