Azure Batch ApplicationPackageReference Error: Resource download failed

Question

Hi,
I am struggling with using Application Package feature for a pool. I have a batch account with an application package which I'd like the deploy to the nodes. I always get the same error on the pool:

Pool Allocation state: Steady
Node state: Unusable

Code: ApplicationPackageError

Message:
One or more application packages specified for the pool are invalid

Values:
ApplicationPackageReference - ffmpeg:4.4
Message - Resource download failed

I have tried different variants defining application package on the job, using a managed identity on the storage account for batch. Using system managed identity on the batch account and give permission on the batch storage account, etc.... it allways ended in the same error message.
Have I something wrong configured? Any hints?
thx, laurent

Batch Account ARM Template:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "batchAccounts_projectapp1dev_name": {
            "defaultValue": "projectapp1dev",
            "type": "String"
        },
        "storageAccounts_projectapp1batchsadev_externalid": {
            "defaultValue": "/subscriptions/..../resourceGroups/project-dev-rg-app1/providers/Microsoft.Storage/storageAccounts/projectapp1batchsadev",
            "type": "String"
        },
        "userAssignedIdentities_project_dev_umid_app1_externalid": {
            "defaultValue": "/subscriptions/...../resourceGroups/project-dev-rg-app1/providers/Microsoft.ManagedIdentity/userAssignedIdentities/project-dev-umid-app1",
            "type": "String"
        }
    },
    "variables": {},
    "resources": [
        {
            "type": "Microsoft.Batch/batchAccounts",
            "apiVersion": "2021-06-01",
            "name": "[parameters('batchAccounts_projectapp1dev_name')]",
            "location": "westeurope",
            "identity": {
                "principalId": "....",
                "tenantId": ".....",
                "type": "SystemAssigned"
            },
            "properties": {
                "autoStorage": {
                    "storageAccountId": "[parameters('storageAccounts_projectapp1batchsadev_externalid')]",
                    "nodeIdentityReference": {
                        "resourceId": "[parameters('userAssignedIdentities_project_dev_umid_app1_externalid')]"
                    },
                    "authenticationMode": "StorageKeys"
                },
                "poolAllocationMode": "BatchService",
                "publicNetworkAccess": "Enabled",
                "encryption": {
                    "keySource": "Microsoft.Batch"
                },
                "allowedAuthenticationModes": [
                    "SharedKey",
                    "AAD",
                    "TaskAuthenticationToken"
                ]
            }
        },
        {
            "type": "Microsoft.Batch/batchAccounts/applications",
            "apiVersion": "2021-06-01",
            "name": "[concat(parameters('batchAccounts_projectapp1dev_name'), '/ffmpeg')]",
            "dependsOn": [
                "[resourceId('Microsoft.Batch/batchAccounts', parameters('batchAccounts_projectapp1dev_name'))]"
            ],
            "properties": {
                "displayName": "ffmpeg",
                "allowUpdates": true,
                "defaultVersion": "4.4"
            }
        },
        {
            "type": "Microsoft.Batch/batchAccounts/applications",
            "apiVersion": "2021-06-01",
            "name": "[concat(parameters('batchAccounts_projectapp1dev_name'), '/sox')]",
            "dependsOn": [
                "[resourceId('Microsoft.Batch/batchAccounts', parameters('batchAccounts_projectapp1dev_name'))]"
            ],
            "properties": {
                "displayName": "sox",
                "allowUpdates": true,
                "defaultVersion": "14.4.2"
            }
        },
        {
            "type": "Microsoft.Batch/batchAccounts/applications/versions",
            "apiVersion": "2021-06-01",
            "name": "[concat(parameters('batchAccounts_projectapp1dev_name'), '/sox/14.4.2')]",
            "dependsOn": [
                "[resourceId('Microsoft.Batch/batchAccounts/applications', parameters('batchAccounts_projectapp1dev_name'), 'sox')]",
                "[resourceId('Microsoft.Batch/batchAccounts', parameters('batchAccounts_projectapp1dev_name'))]"
            ],
            "properties": {}
        },
        {
            "type": "Microsoft.Batch/batchAccounts/applications/versions",
            "apiVersion": "2021-06-01",
            "name": "[concat(parameters('batchAccounts_projectapp1dev_name'), '/ffmpeg/4.4')]",
            "dependsOn": [
                "[resourceId('Microsoft.Batch/batchAccounts/applications', parameters('batchAccounts_projectapp1dev_name'), 'ffmpeg')]",
                "[resourceId('Microsoft.Batch/batchAccounts', parameters('batchAccounts_projectapp1dev_name'))]"
            ],
            "properties": {}
        },
        {
            "type": "Microsoft.Batch/batchAccounts/pools",
            "apiVersion": "2021-06-01",
            "name": "[concat(parameters('batchAccounts_projectapp1dev_name'), '/app1Pool')]",
            "dependsOn": [
                "[resourceId('Microsoft.Batch/batchAccounts', parameters('batchAccounts_projectapp1dev_name'))]",
                "[resourceId('Microsoft.Batch/batchAccounts/applications', parameters('batchAccounts_projectapp1dev_name'), 'ffmpeg')]"
            ],
            "properties": {
                "vmSize": "STANDARD_A2",
                "interNodeCommunication": "Disabled",
                "taskSlotsPerNode": 1,
                "taskSchedulingPolicy": {
                    "nodeFillType": "Pack"
                },
                "deploymentConfiguration": {
                    "virtualMachineConfiguration": {
                        "imageReference": {
                            "publisher": "microsoftwindowsserver",
                            "offer": "windowsserver",
                            "sku": "2019-datacenter-core",
                            "version": "latest"
                        },
                        "nodeAgentSkuId": "batch.node.windows amd64",
                        "windowsConfiguration": {
                            "enableAutomaticUpdates": false
                        },
                        "nodePlacementConfiguration": {
                            "policy": "Regional"
                        }
                    }
                },
                "networkConfiguration": {
                    "publicIPAddressConfiguration": {
                        "provision": "BatchManaged"
                    }
                },
                "scaleSettings": {
                    "fixedScale": {
                        "targetDedicatedNodes": 0,
                        "targetLowPriorityNodes": 1,
                        "resizeTimeout": "PT15M"
                    }
                },
                "applicationPackages": [
                    {
                        "id": "[resourceId('Microsoft.Batch/batchAccounts/applications', parameters('batchAccounts_projectapp1dev_name'), 'ffmpeg')]",
                        "version": "4.4"
                    }
                ]
            }
        }
    ]
}

Answer 1

@Laurent Christen Apologies for the delay in response and all the inconvenience caused because of the issue.

I had discussion internally and below are the inputs I got:

It seems like It looks like the customer never configured the managed identity to use on the pool therefore meaning there is no identity to use. Can you please follow the below and see if you are still facing the issue:

https://learn.microsoft.com/en-us/azure/batch/managed-identity-pools

Hope it helps!!!

Please "Accept as Answer" if it helped so it can help others in community looking for help on similar topics.