App Config Private DNS Zone getting deleted automatically by "Azure Traffic Manager and DNS"

Apurva Pathak 570

Hi all, Happy New Year! I have created private endpoint of App Config service but the A record in Private DNS Zone getting deleted automatically by "Azure Traffic Manager and DNS" Any help on this will be greatly appreciated! Thanks in advance! Creation of A record User's image

Deletion of A record User's image

Ishmeet Kaur 75 Reputation points Microsoft Employee

2024-01-15T09:55:29.91+00:00
Hi @Apurva Pathak , kindly check if any of the mentioned scenarios apply to your problem:

Were there any changes made to the Private Endpoint connection (removed or rejected from PaaS service or PaaS service was deleted) during this timestamp? Kindly check the connection status of Private Endpoint and confirm.

Do you have any other Private Endpoints for same PaaS service which are linked to the same private DNS zone and that Private Endpoint was deleted at the time of this operation?
Ishmeet Kaur 75 Reputation points Microsoft Employee

2024-01-18T04:20:14.38+00:00

Hi @Apurva Pathak checking if you got a chance to look at my previous comment. Your response is highly appreciated!
Apurva Pathak 570 Reputation points

2024-01-18T06:53:11.5666667+00:00

Hi Ishmeet, sorry for delay. I did check that and nothing as such is happening. Could it be caused by two policies conflicting with each other?
Ishmeet Kaur 75 Reputation points Microsoft Employee

2024-01-18T07:03:21.62+00:00

@Apurva Pathak could you please clarify your ask. Which policies are you referring to?
Apurva Pathak 570 Reputation points

2024-01-18T07:13:17.4166667+00:00

We have deployed policies to automatically integrate DNS zones with PEs.
Ishmeet Kaur 75 Reputation points Microsoft Employee

2024-01-18T07:38:06.92+00:00

Yes, two conflicting policies can cause this problem if two Private Endpoints for same PaaS service are getting integrated with same private DNS zone. I would recommend opening a Support Request so that the Support Team can check the Private Endpoints and Private DNS Zone configuration to confirm the root cause of the record deletion issue.
Apurva Pathak 570 Reputation points

2024-01-18T07:44:52.2+00:00

Sure, thanks!
GitaraniSharma-MSFT 49,626 Reputation points Microsoft Employee

2024-01-18T13:14:09.04+00:00
Hello @Apurva Pathak ,

As mentioned by @Ishmeet Kaur, this issue would require a deeper investigation, so if you have a support plan, I request you file a support ticket, else please do let us know, we will try and help you get a one-time free technical support.

In case you need help with a one-time free technical support, I would request you to send an email with subject line "ATTN gishar | App Config Private DNS Zone getting deleted automatically by Azure Traffic Manager and DNS" to AzCommunity[at]Microsoft[dot]com with the following details, I will follow-up with you.

Reference this Q&A thread

Your Azure Subscription ID

Note: Do not share any PII data as a public comment.

We will post a summarized answer once the issue is resolved.

Regards,

Gita
GitaraniSharma-MSFT 49,626 Reputation points Microsoft Employee

2024-01-24T14:14:33.4866667+00:00

@Apurva Pathak , could you please provide an update on this post?
GitaraniSharma-MSFT 49,626 Reputation points Microsoft Employee

2024-02-01T12:45:57.7766667+00:00

@Apurva Pathak , do you have any updates on this issue?
Apurva Pathak 570 Reputation points

2024-02-01T12:53:02.9266667+00:00

@GitaraniSharma-MSFT Apologies for delay in getting back to you! We have raised case, we can disregard this.
GitaraniSharma-MSFT 49,626 Reputation points Microsoft Employee

2024-02-01T12:56:21.4233333+00:00

@Apurva Pathak , thank you for the update.
Eric Ehret 0 Reputation points

2024-07-15T16:11:47.47+00:00

We have seen the same thing happen. In our case it was for a KeyVault.
This built-in policy has been used to automatically configure the Private DNS zone integration.
/providers/Microsoft.Authorization/policyDefinitions/ac673a9a-f77d-4846-b2d8-a57f8e1c01d4

For the most part that has been working fine, but this past week the A record was inexplicably deleted from the zone even though the private endpoint itself and the key vault both remained.
Even the DNS configuration in the Private Endpoint was still intact... it's just the A record that disappeared.

Would love to hear if anyone else discovered the cause of this issue and knows the solution!
Evan Venell 0 Reputation points

2024-07-29T11:53:05.08+00:00

Has there been any update on this issue? We have also been experiencing the A record being inexplicably deleted without ever actually changing anything in our normal deployments.
Apurva Pathak 570 Reputation points

2024-07-29T15:06:49.9733333+00:00

Hi @Evan Venell @Eric Ehret ,

Could you please help me with the resource which has initiated this removal in DNS Zone. Also, please have a look at the activity logs of PEs and the NICs.

Thanks!

1 answer

Eric Ehret 0 Reputation points

2024-07-29T21:21:53.1833333+00:00

Nevermind -

After digging into the activity logs as suggested, I discovered that there was a suspicious entry relating to the private endpoint connection proxy getting overwritten.

Apparently someone was attempting to create an additional extraneous private endpoint for the same resource, creating a conflict. It seems that when that happens "Azure Traffic Manager and DNS" doesn't recover well; even though the extraneous private endpoint did not persist and ended up getting removed again, the change of the private endpoint connection proxy due to the attempted configuration of the extra PE seems to have caused "Azure Traffic Manager and DNS" to remove the "incorrect" A record. But then it did not replace it with anything

This is my best effort to understand the situation at the moment. I may not have explained it 100% correctly, but this is how it looks based on the logs I've seen.

My focus is going to now be on helping explain to this user why the second PE is not required, to avoid attempting to create that, and then resolve the separate networking issue that was prompting them to do this in the first place.
Please sign in to rate this answer.
Evan Venell 0 Reputation points

2024-08-02T15:35:10.45+00:00

Hi Eric,

Yeah that's similar to my circumstance. We utilize bicep to do these deployments and I think something changed with the way Microsoft is handling the relationship between PE's and Private DNS Zones. Specifically, per environment so depending on your networking setup if you had some kind of automation templated deployment like bicep (just speculating if it's similar to our circumstance) the bicep template (which only created the app config and a PE connection at the time) tried to spin up a new privatednszone for the wrong environment (which was a lower environment I was deploying to) which ripped out the A record out of the prod private dns zone and into a newly created one. All that to say thats where I followed the same trail as to why the Azure Traffic Montior & DNS event source deleted that A record. Hope you find out more soon, we are still troubleshooting on our end how to resolve this in an automated way.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

App Config Private DNS Zone getting deleted automatically by "Azure Traffic Manager and DNS"

1 answer

Your answer