azure private dns zone: public name resolution is not being intercepted

Question

azure private dns zone: public name resolution is not being intercepted

Alexander Butz 25

Hi all.

Setup

I have a vnet with multiple subnets, one for private endpoints, one for a APIM. APIM is fully vnet integrated in developer Tier (stv2.1). I want to do requests against a Service Bus Namespace (Premium) within one APIM API. All subnets have "Private endpoint network policy" set to "disabled". Service Bus Namespace has public access disabled. Private Endpoints show Provisioning state "Succeeded", Connection status "Approved" and Request/Response "Auto-Approved". I created a private DNS zone for each of my resources that are using a private endpoint, for example KeyVault and Service Bus Namespace with names privatelink.vaultcore.azure.net and privatelink.servicebus.windows.net. I created the dns entries in there using bicep. A records that point to the IP of the corresponding private endpoint.

expected behaviour

When resolving dns names using "Resolve-DnsName" on a windows VM that is hosted within the APIM subnet the IP of the private endpoint of the corresponding resource should be returned.

actual behaviour

When resolving dns names using "Resolve-DnsName" on a windows VM that is hosted within the APIM subnet the response does contain the CNAME record which points to the .privatelink.servicebus.windows.net domain but there is no private IP being returned. Only the public IP of the public domain.

things I tried

recreated link of private dns zone to vnet
recreated dns records in private dns zone
recreated entire private endpoint of SBNS

testing

PS C:\Users\temp-vm> Resolve-DnsName "<SBNS_Name>.servicebus.windows.net"

Name                           Type   TTL   Section    NameHost
----                           ----   ---   -------    --------
<SBNS_Name>.servicebus.windows.net CNAME  30    Answer     <SBNS_Name>.privatelink.servicebus.windows.net

<SBNS_Name>.privatelink.servicebus.windows.net CNAME  30    Answer     ns-prod-fra21-401.germanywestcentral.cloudapp.azure.com


Name       : ns-prod-fra21-401.germanywestcentral.cloudapp.azure.com
QueryType  : A
TTL        : 10
Section    : Answer
IP4Address : <public IP SBNS>


Name                   : germanywestcentral.cloudapp.azure.com
QueryType              : SOA
TTL                    : 202
Section                : Authority
NameAdministrator      : azuredns-hostmaster.microsoft.com
SerialNumber           : 1
TimeToZoneRefresh      : 3600
TimeToZoneFailureRetry : 300
TimeToExpiration       : 2419200
DefaultTTL             : 300



PS C:\Users\temp-vm> Resolve-DnsName "<SBNS_Name>.privatelink.servicebus.windows.net"

Name                           Type   TTL   Section    NameHost
----                           ----   ---   -------    --------
<SBNS_Name>.privatelink.servicebus.windows.net CNAME  22    Answer     ns-prod-fra21-401.germanywestcentral.cloudapp.azure.com


Name       : ns-prod-fra21-401.germanywestcentral.cloudapp.azure.com
QueryType  : A
TTL        : 2
Section    : Answer
IP4Address : <public IP SBNS>



PS C:\Users\temp-vm> nslookup "<SBNS_Name>.servicebus.windows.net"
Server:  UnKnown
Address:  168.63.129.16

Non-authoritative answer:
Name:    ns-prod-fra21-401.germanywestcentral.cloudapp.azure.com
Address:  <public IP SBNS>
Aliases:  <SBNS_Name>.servicebus.windows.net
          <SBNS_Name>.privatelink.servicebus.windows.net



PS C:\Users\temp-vm> Resolve-DnsName "<KV_Name>.vault.azure.net"

Name                           Type   TTL   Section    NameHost
----                           ----   ---   -------    --------
<KV_Name>.vault.azure.net CNAME  60    Answer     <KV_Name>.privatelink.vaultcore.azure.net


Name       : <KV_Name>.privatelink.vaultcore.azure.net
QueryType  : A
TTL        : 10
Section    : Answer
IP4Address : 10.250.134.4

As you can see, the exact same setup works fine for the Key Vault private endpoint, as well as for other resources like a storage account's blob storage (not listed here). I really don’t understand why I’m not getting the private IP in this case. Any help in understanding this would be greatly appreciated!

Sai Prabhu Naveen Parimi 2,510 Reputation points Microsoft External Staff Moderator

2025-05-20T13:00:42.73+00:00
@Alexander Butz

Here are a couple of follow-up questions that could help in narrowing down the issue:

Were there any specific configurations or steps that you used when setting up the private endpoint for the Service Bus that differ from the other services that are working correctly?

Can you confirm if the private DNS records and zone were set up after the private endpoint was created?

Have you checked if there are any firewall rules or network security groups that might be affecting the DNS resolution from the APIM subnet to the private endpoint?
Alexander Butz 25 Reputation points

2025-05-20T13:09:29.2966667+00:00
@Sai Prabhu Naveen Parimi thank you for your fast response.

No. All private dns zones as well as PEs are based on the same correspnding bicep module.

Yes they are present.

There was one route table with a route for internet facing traffic from APIM subnet to a Azure Firewall within the same VNet. I changed this roule from destination Firewall to destination Internet .
Bodapati Harish 820 Reputation points Microsoft External Staff Moderator

2025-05-20T17:44:52.6633333+00:00

Hello Alexander Butz,

Thank you for the update.

The issue appears to be caused by a user-defined route that was directing internet-bound traffic from the APIM subnet to Azure Firewall. This routing setup was intercepting DNS queries and causing the resolution to fall back to the public domain instead of returning the private endpoint IP address. After modifying the route to direct traffic to the Internet instead of the firewall, the DNS resolution began returning the expected private IP, which confirms that the private endpoint and private DNS zone configurations are correct.

In scenarios where routing through Azure Firewall is still required, such as for traffic inspection or compliance, consider enabling DNS proxy on Azure Firewall. This setting ensures that DNS queries are handled correctly and private resolution is maintained. Azure Firewall DNS proxy overview.

Alternatively, using a DNS forwarder like a custom DNS server or Azure DNS Private Resolver can help manage internal DNS resolution without relying on public DNS paths. Azure DNS Private Resolver.
Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-05-21T00:56:40.5366667+00:00

Hello Alexander Butz, could you please check your private message and respond there?
Alexander Butz 25 Reputation points

2025-05-21T07:11:05.1833333+00:00

@Bodapati Harish I removed this route to the firewall yesterday but the issue still persists. I cant get the private IP, not from a VM in the APIM subnet and not from a VM within PE subnet. Since the route from specific subnet to firewall was only present for apim subnet and not for my PE subnet I dont think this route was/will be the issue.
Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-05-21T07:27:35.58+00:00

Hello Alexander Butz, could you please check your private message and respond there?
Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-05-22T13:06:11.2833333+00:00

Hello Alexander Butz,Could you please confirm whether the VNet is linked to the Private DNS zone, and also share a screenshot of the virtual network links?

Accepted answer

2 additional answers

Your answer

Sai Prabhu Naveen Parimi 2,510 Reputation points Microsoft External Staff Moderator

2025-05-20T13:00:42.73+00:00

@Alexander Butz

Here are a couple of follow-up questions that could help in narrowing down the issue:

Were there any specific configurations or steps that you used when setting up the private endpoint for the Service Bus that differ from the other services that are working correctly?

Can you confirm if the private DNS records and zone were set up after the private endpoint was created?

Have you checked if there are any firewall rules or network security groups that might be affecting the DNS resolution from the APIM subnet to the private endpoint?
Alexander Butz 25 Reputation points

2025-05-20T13:09:29.2966667+00:00

@Sai Prabhu Naveen Parimi thank you for your fast response.

No. All private dns zones as well as PEs are based on the same correspnding bicep module.

Yes they are present.

There was one route table with a route for internet facing traffic from APIM subnet to a Azure Firewall within the same VNet. I changed this roule from destination Firewall to destination Internet .
Bodapati Harish 820 Reputation points Microsoft External Staff Moderator

2025-05-20T17:44:52.6633333+00:00

Hello Alexander Butz,

Thank you for the update.

The issue appears to be caused by a user-defined route that was directing internet-bound traffic from the APIM subnet to Azure Firewall. This routing setup was intercepting DNS queries and causing the resolution to fall back to the public domain instead of returning the private endpoint IP address. After modifying the route to direct traffic to the Internet instead of the firewall, the DNS resolution began returning the expected private IP, which confirms that the private endpoint and private DNS zone configurations are correct.

In scenarios where routing through Azure Firewall is still required, such as for traffic inspection or compliance, consider enabling DNS proxy on Azure Firewall. This setting ensures that DNS queries are handled correctly and private resolution is maintained. Azure Firewall DNS proxy overview.

Alternatively, using a DNS forwarder like a custom DNS server or Azure DNS Private Resolver can help manage internal DNS resolution without relying on public DNS paths. Azure DNS Private Resolver.
Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-05-21T00:56:40.5366667+00:00

Hello Alexander Butz, could you please check your private message and respond there?
Alexander Butz 25 Reputation points

2025-05-21T07:11:05.1833333+00:00

@Bodapati Harish I removed this route to the firewall yesterday but the issue still persists. I cant get the private IP, not from a VM in the APIM subnet and not from a VM within PE subnet. Since the route from specific subnet to firewall was only present for apim subnet and not for my PE subnet I dont think this route was/will be the issue.
Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-05-21T07:27:35.58+00:00

Hello Alexander Butz, could you please check your private message and respond there?
Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-05-22T13:06:11.2833333+00:00

Hello Alexander Butz,Could you please confirm whether the VNet is linked to the Private DNS zone, and also share a screenshot of the virtual network links?

Answer 1

Hi @Alexander Butz ,

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this!

Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others", I'll repost your solution in case you'd like to Accept the answer.

Issue:

When attempting to access a Service Bus Namespace (Premium) via Private Endpoint from a VNet-integrated APIM instance, DNS resolution was not returning the private IP.

Despite having correct configurations for Private DNS Zones and Private Endpoints (all showing Approved and Succeeded states), DNS queries resolved to the public IP rather than the private one. The issue persisted even after verifying route tables, NSGs, and the correctness of DNS zone names and records.

Solution:

The root cause was traced to a Private DNS Resolver that was connected to a peered VNet. According to Azure Support, if this DNS Resolver is not configured correctly, it can interfere with private DNS resolution, even in other connected VNets.

Once this misconfigured DNS Resolver was identified, it became clear why the correct private IPs were not being returned for specific services (like Service Bus), while others (like Key Vault) were resolving correctly.

Please click Accept Answer and kindly upvote it so that other people who faces similar issue may get benefitted from it.

Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-05-29T11:57:43.0766667+00:00

Hi @Alexander Butz , please click Accept Answer and kindly upvote it so that other people who faces similar issue may get benefited from it.

Answer 2

Alex Burlachenko 10,565

Hi Alexander,

Thanks for your patience while we work through this tricky issue.

Since you've confirmed the route table change didn't help and the problem affects both subnets equally, let's focus on some Service Bus-specific behaviors that might explain what's happening.

One thing that's different about Service Bus compared to Key Vault is how it handles DNS resolution. When you create a private endpoint for Service Bus, Azure actually creates two DNS records:

A CNAME record pointing to the privatelink address

An A record with the private IP in your privatelink zone

The issue you're seeing where it resolves to the public IP suggests the private DNS zone isn't being properly consulted during name resolution. Here are some specific things to check:

First, could you verify if the private endpoint's network interface has the correct IP address? You can find this by:

Going to your private endpoint resource in Azure portal

Clicking on the network interface under "Network Interface"

Checking that the private IP matches what's in your privatelink.servicebus.windows.net DNS zone

Sometimes there can be a mismatch here that causes resolution to fail. The Microsoft docs explain this in more detail: Private Endpoint DNS Configuration

Another test worth trying is to query the private DNS zone directly from your VM:

Resolve-DnsName "<SBNS_Name>.privatelink.servicebus.windows.net"

If this returns the correct private IP but the regular servicebus.windows.net name doesn't, it confirms your private zone is working but the linkage between the public and private names isn't happening properly.

One last thought - have you checked if there are any network security groups (NSGs) that might be blocking DNS traffic to 168.63.129.16? This is Azure's internal DNS resolver that handles the private zone lookups. Even though you've checked routing, sometimes NSG rules can interfere.

I know this is a lot to check, but we'll get to the bottom of it! Let me know what you find with these additional tests.

Best regards,
Alex
P.S. If my answer help to you, please Accept my answer
PPS That is my Answer and not a Comment
https://ctrlaltdel.blog/

Alexander Butz 25 Reputation points

2025-05-23T07:34:16.86+00:00

Hi @Alex Burlachenko ,

"First, could you verify if the private endpoint's network interface has the correct IP address? You can find this by:"
Its the correct IP address. I also created a new PE with another IP and created a new DNS record without change in the DNS behavior.

"Another test worth trying is to query the private DNS zone directly from your VM:"

This does return the private IP.

"One last thought - have you checked if there are any network security groups (NSGs) that might be blocking DNS traffic to 168.63.129.16?"

The NSG on the subnet in question has only the ruleset for APIM implemented. Outgoing as well as ingoing rules do only allow traffic, the only rule that denies traffic is the default 65500 "denyAllOutBound".
Alex Burlachenko 10,565 Reputation points

2025-05-23T08:01:56.35+00:00

Hi,

The key thing we've discovered is that when you query the privatelink address directly, it correctly returns the private IP, but when you use the regular servicebus.windows.net address, it's still giving you the public one. This tells us your private DNS zone is working fine, but Azure's DNS system isn't properly "catching" the request to redirect it to your private zone like it should.

One thing that often causes this is a tiny mismatch in the private DNS zone name. Could you check that your zone is named exactly "privatelink.servicebus.windows.net" (no typos, no extra characters)? Even something as small as missing the 's' in "windows" would make it not work. The Microsoft docs show the exact naming needed here: Private DNS zone naming requirements.

Another thing to look at is whether the virtual network link to your private DNS zone might have gotten into a weird state. Sometimes recreating this link can help - you'd remove the existing link and then add it back fresh. Make sure it's linked to the VNet where your VMs are running, and that the "Enable auto-registration" box is unchecked since you're managing records manually. Here's how to check this: Managing VNet links to private zones.

Since you mentioned the NSGs look good and recreating the private endpoint didn't help, I'm wondering if there might be some DNS caching happening at the platform level. As a test, you could try creating a brand new private DNS zone with a slightly different name (like "privatelink2.servicebus.windows.net"), link it to your VNet, add the A record, and see if that works any differently. Sometimes starting fresh with new resources helps bypass any hidden caching issues.

Also, just to cover all bases - when you check the private endpoint connection in your Service Bus namespace, does it show as "Approved" with the status "Succeeded"? Occasionally there can be a delay in the backend propagation even when everything looks right in the portal. The Microsoft docs mention this can take a few minutes: Private endpoint connection states.

I know this feels like we're going in circles, but we're actually making progress by eliminating possibilities. Let me know what you find with these additional checks - I'm confident we'll get this sorted out soon!

rgds,

Alex
Alexander Butz 25 Reputation points

2025-05-23T08:10:04.5866667+00:00

"One thing that often causes this is a tiny mismatch in the private DNS zone name. Could you check that your zone is named exactly "privatelink.servicebus.windows.net" (no typos, no extra characters)?"

Checked. Name is exactly "privatelink.servicebus.windows.net".

"Another thing to look at is whether the virtual network link to your private DNS zone might have gotten into a weird state."

I recreated this link already multiple times using the bicep template that was initially used to create these resources and also by hand within the portal to make sure there is no misconfiguration within the bicep template.

"As a test, you could try creating a brand new private DNS zone with a slightly different name (like "privatelink2.servicebus.windows.net"), link it to your VNet, add the A record, and see if that works any differently. Sometimes starting fresh with new resources helps bypass any hidden caching issues."

But what do we achive doing this? A privatelink2 private dns zone would never be used unless I query for .privatelink2. domain directly. Since this is the one part thats working I dont understand whats the benefit of this.

"Also, just to cover all bases - when you check the private endpoint connection in your Service Bus namespace, does it show as "Approved" with the status "Succeeded"?"

PE shows "Connection state" "Approved" and "Provisioning state" "Succeeded".
Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-05-23T08:22:13.34+00:00

Hello Alexander Butz, Could you please confirm from where you are trying to access the service Bus — is it from an Azure VM within the same VNet or from outside the VNet?
Alexander Butz 25 Reputation points

2025-05-23T08:48:00.89+00:00

The VM I am using for testing is in the same subnet as the APIM from which I want to access the SBNS via private Endpoint. The PE of SBNS is placed in another subnet which is part of the same VNet.
Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-05-26T13:51:56.19+00:00

Hello Alexander Butz, Could you please share the result from APIM?
Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-05-27T16:44:30.1133333+00:00

Hello @Alexander Butz, could you please check your private message and respond there?

Answer 3

Alexander Butz 25

We found a private DNS resolver which is connected to one of the peered VNets. According to Azure Support, this can interfere with private DNS if it is not configured correctly.

Share via

azure private dns zone: public name resolution is not being intercepted

Setup

expected behaviour

actual behaviour

things I tried

testing

2 additional answers

Your answer