Azure Backup and Private Endpoints (UserErrorNetworkAccessPolicyIsDenyAll:Network access policy is set to deny all)

Question

Hi All,

we are looking to configure an Azure Recovery Services Vault which will be used for both Azure VM backups as well as (later) for MARS agent file-based backups of some on-prem servers - see below for the bicep template.

Due to the latter we want to configure a private endpoint and disable Public Network Access. (If I read the docs right this should not make a difference to the Azure VM backup as it essentially operates in the backplane of Azure and doesnt need any public or private endpoint at all). I also understand from the docs that its really important to create the private endpoint first before any backup workloads (which is what we did).

We have the Recovery Services Vault and the Private Endpoint configured:

The Private Endpoint has Connection State "Approved"
Public Network access of the vault is denied (kind of the point of a private endpoint)
I removed all NSGs for debug purposes
The effective route of the Azure VM routes to the private endpoint
DNS zone integration has worked and entries were added to the privatelink.we.backup.windowsazure.com zone which is resolvable from the VM
Per the docs this setup should auto-create private endpoints for storage and queues. This has not happened yet, however, I understand this only happens when the first workload is created.

I am now trying to create (manually in the portal) my first VM backup. This consistently fails deplpyment with

code":"UserErrorNetworkAccessPolicyIsDenyAll","message":"Network access policy is set to deny all"

This has the sound as if it doesnt like the fact that Public Network Access is denied. However for testing purposes I set Public network access back to "Allow from all networks" and the deployment still fails.

It seems we have created an essentially unusable Recovery Services Vault no matter the network settings.

Any idea what could be wrong?

Kind Regards

Jens

Bicep for the vault:

resource rsv_prdsec_westeu_001 'Microsoft.RecoveryServices/vaults@2023-02-01' = {
  name: vaultName
  location: location
  tags: tags
  sku: {
    name: 'RS0'
    tier: 'Standard'
  }
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    monitoringSettings: {
      azureMonitorAlertSettings: {
        alertsForAllJobFailures: 'Enabled'
      }
      classicAlertSettings: {
        alertsForCriticalOperations: 'Disabled'
      }
    }
    securitySettings: {}
    publicNetworkAccess: 'Disabled'
    restoreSettings: {
      crossSubscriptionRestoreSettings: {
        crossSubscriptionRestoreState: 'Enabled'
      }
    }
  }
}

and here is the private endpoint as well as the DNS Zone integration:

resource pep_rsv_prdsec_westeu_001 'Microsoft.Network/privateEndpoints@2022-09-01' = {
  name: 'pep-rsv-prdsec-westeu-001'
  location: location
  tags: tags
  properties: {
    privateLinkServiceConnections: [
      {
        name: 'pep-rsv-prdsec-westeu-001'
        properties: {
          privateLinkServiceId: rsv_prdsec_westeu_001.id
          groupIds: [
            'AzureBackup'
          ]
        }
      }
    ]
    manualPrivateLinkServiceConnections: []
    subnet: {
      id: privateEndpointSNetID
    }
    ipConfigurations: [  ]
    customDnsConfigs: [  ]   
  }
}


resource backupDNSZone 'Microsoft.Network/privateDnsZones@2020-06-01' existing = {
  name: 'privatelink.we.backup.windowsazure.com'
  scope: resourceGroup('XXXXXXXXXXXXXXXXXX','rg-connectivity-westeu-002') 
}

resource queueDNSZone 'Microsoft.Network/privateDnsZones@2020-06-01' existing = {
  name: 'privatelink.queue.core.windows.net'
  scope: resourceGroup('XXXXXXXXXXXXXXXXX','rg-connectivity-westeu-002') 
}

resource blobDNSZone 'Microsoft.Network/privateDnsZones@2020-06-01' existing = {
  name: 'privatelink.blob.core.windows.net'
  scope: resourceGroup('XXXXXXXXXXXXXXXX','rg-connectivity-westeu-002') /
}


resource zoneGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2022-09-01'= {
  name: 'backupdnsgroup'
  parent: pep_rsv_prdsec_westeu_001
  properties: {
    privateDnsZoneConfigs: [
      {
        name: 'config1'
        properties: {
          privateDnsZoneId: backupDNSZone.id
        }
      }
      {
        name: 'config2'
        properties: {
          privateDnsZoneId: queueDNSZone.id
        }
      }
      {
        name: 'config3'
        properties: {
          privateDnsZoneId: blobDNSZone.id
        }
      }
    ]
  }
}

and here is the backup the portal wants to submit:

{
    "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "armProviderNamespace": {
            "type": "String"
        },
        "vaultName": {
            "type": "String"
        },
        "vaultRG": {
            "type": "String"
        },
        "vaultSubID": {
            "type": "String"
        },
        "policyName": {
            "type": "String"
        },
        "fabricName": {
            "type": "String"
        },
        "protectionContainers": {
            "type": "Array"
        },
        "protectedItems": {
            "type": "Array"
        },
        "sourceResourceIds": {
            "type": "Array"
        },
        "extendedProperties": {
            "type": "Array"
        }
    },
    "resources": [
        {
            "type": "Microsoft.RecoveryServices/vaults/backupFabrics/protectionContainers/protectedItems",
            "apiVersion": "2016-06-01",
            "name": "[concat(parameters('vaultName'), '/', parameters('fabricName'), '/',parameters('protectionContainers')[copyIndex()], '/', parameters('protectedItems')[copyIndex()])]",
            "properties": {
                "protectedItemType": "Microsoft.ClassicCompute/virtualMachines",
                "policyId": "[resourceId(concat(parameters('armProviderNamespace'), '/vaults/backupPolicies'), parameters('vaultName'), parameters('policyName'))]",
                "sourceResourceId": "[parameters('sourceResourceIds')[copyIndex()]]",
                "extendedProperties": "[parameters('extendedProperties')[copyIndex()]]"
            },
            "copy": {
                "name": "protectedItemsCopy",
                "count": "[length(parameters('protectedItems'))]"
            }
        }
    ]
}

and last the parameters the portal filled in:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "armProviderNamespace": {
            "value": "Microsoft.RecoveryServices"
        },
        "vaultName": {
            "value": "rsv-prdsec-westeu-001"
        },
        "vaultRG": {
            "value": "rg-prdsec-weu-001"
        },
        "vaultSubID": {
            "value": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
        },
        "policyName": {
            "value": "bp-prdsec-vm-westeu-001"
        },
        "fabricName": {
            "value": "Azure"
        },
        "protectionContainers": {
            "value": [
                "IaasVMContainer;iaasvmcontainerv2;rg-prdsec-weu-001;vmtrakaprdweu01"
            ]
        },
        "protectedItems": {
            "value": [
                "vm;iaasvmcontainerv2;rg-prdsec-weu-001;vmtrakaprdweu01"
            ]
        },
        "sourceResourceIds": {
            "value": [
                "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-prdsec-weu-001/providers/Microsoft.Compute/virtualMachines/vmtrakaprdweu01"
            ]
        },
        "extendedProperties": {
            "value": [
                {}
            ]
        }
    }
}

Accepted Answer

@Anonymous Thank you for the detailed information. Can you please let me know if you are backing up your azure vm using Enhanced policy ? If so, your backup operation using enhanced policy will fail due to network access policy settings. Only disks with ‘Public endpoint’ are supported currently (for backup using enhanced policy). To resolve this issue, Navigate to 'Disk setting > Networking > Network connectivity method' and set connectivity method to "Public Endpoint" for enabling backups with Enhanced Policy.

Azure Backup and Private Endpoints (UserErrorNetworkAccessPolicyIsDenyAll:Network access policy is set to deny all)

0 additional answers