Application Gateway and Key Vault only accessible using Private Endpoints (policy requirement)

Question

So, we have an Azure Key Vault where the Application Gateway shall retrieve the SSL Certificates it needs to handle HTTPS traffic. This Key Vault is restricted (due to company policies) to only be accessible using private endpoints.

According to the documentation this should be possible, but it isn't I even tried the strange recommendation to connect the key vault private DNS zone to the virtual network where the application gateway has its subnet to no avail.

The key vault audit logs shows quite clearly that it can't be accessed due to "Connection is not an approved private link and caller was ignored because bypass is not set to 'AzureServices' and PublicNetworkAccess is set to 'Disabled'.". I can even see that AGW tries to connect with an Azure internal IP address and not with any IP originating from the attached GW subnet.

So, is this simply the fact that the documentation is wrong? It isn't possible that to enforce the traffic over private endpoints and the only solution is to bypass Azure Service traffic?

Answer 1

Hi @ Rune Gulbrandsen,

Welcome to Microsoft Q&A Platform.

Application Gateway cannot retrieve certificates from Azure Key Vault through a private endpoint. The supported configuration is to allow trusted Microsoft services to bypass the Key Vault firewall.

According to the Key Vault networking documentation, enabling “Allow trusted Microsoft services to bypass this firewall” allows specific Azure services to bypass the network restrictions applied to the Key Vault.

Please refer Allow trusted Microsoft services to bypass Key Vault firewall

As of March 15, 2021, Azure Key Vault recognizes Application Gateway as a trusted service by leveraging User-Assigned Managed Identities for authentication to Azure Key Vault. When combined with service endpoints and enabling the trusted services option in the Key Vault firewall settings, Application Gateway can successfully access certificates stored in Key Vault., Refer : Verify Firewall Permissions to Key Vault

As Jack Stromberg mentioned here: Azure Application Gateways do not resolve Private Endpoints of Keyvault

If Key Vault is being used as a backend target (part of your backend pool), specify the <yourvault>.privatelink.vaultcore.azure.net address as the fqdn for the backend target. Within your corresponding backend HTTP Setting, configure Override with new host name with the value of yes and check Override with specific domain name for Host name override. For the hostname to override, use the FQDN provided by keyvault (i.e. <yourvault>.vault.azure.net).

Pleaseand “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

I'll answer this myself here.

Application Gateway traffic to retrieve SSL certificates to be used by the frontend listener does not support retrieving certificates from Key Vaults that are only accessible using private endpoint, even though the documentation says it does.

You need to at least enable "Allow public access from specific virtual networks and IP addresses" and enable the "Allow trusted Microsoft services to bypass this firewall" for the Key Vault as Application Gateway only uses the Azure internal networking to fetch the certificates.

Answer 3

That private DNS zone configuration is required - for details, refer to https://learn.microsoft.com/en-us/answers/questions/988883/azure-application-gateways-do-not-resolve-private

"If Key Vault is being used as a backend target (part of your backend pool), specify the <yourvault>.privatelink.vaultcore.azure.net address as the fqdn for the backend target. Within your corresponding backend HTTP Setting, configure Override with new host name with the value of yes and check Override with specific domain name for Host name override. For the hostname to override, use the FQDN provided by keyvault (i.e. <yourvault>.vault.azure.net).

If Key Vault is being referenced via private endpoint for a listener, you must associate the private dns zone to the virtual network."

Beyond that, note that Azure Application Gateway utilizes a split-DNS logic where its management plane traffic - specifically the process of fetching SSL certificates from Key Vault - and does not always follow the same routing rules as your standard backend traffic. When AGW attempts to retrieve a certificate, it ignores your custom DNS settings and fall back to the Azure default resolver, which resolves the Key Vault to its public IP address even if a Private Endpoint is present. Because your company policy has disabled public network access on the Key Vault, this public-bound request is immediately rejected by the firewall, resulting in the audit log error you are seeing.

The "internal IP" you see in your logs indicates that the Application Gateway is reaching out as a "Trusted Microsoft Service" from the Azure backbone rather than appearing as a client from its own delegated subnet. To satisfy your company's restriction while allowing this connection, you should enable the "Allow trusted Microsoft services to bypass this firewall" setting in the Key Vault's networking configuration. This specific bypass is designed for exactly this scenario: it allows internal platform operations, like AGW certificate fetching, to reach the vault securely even when public access is otherwise blocked.

If your policy is so restrictive that it forbids the "Trusted Services" bypass, the remaining path is to ensure perfect Private Link DNS resolution, which tends to be tricky with AGW. For this to work, the privatelink.vaultcore.azure.net DNS zone should be linked directly to the virtual network where the Application Gateway resides, and the gateway should be restarted (stopped and then started) to clear any cached DNS lookups. However, even with these steps, if the AGW continues to use its internal management IP to call the vault, the Key Vault will still block the connection unless that bypass is active or the AGW's specific subnet is whitelisted via Service Endpoints.

Effectively, the most reliable approach would be to combine the Private Endpoint with the "Trusted Microsoft Services" bypass. This allows the Key Vault to remain closed to the public internet while permitting the Application Gateway's internal identity to authenticate and retrieve the necessary SSL certificates over the Microsoft backbone.

---

If the above response helps answer your question, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.

hth

Marcin