Failover Cluster DNS error, event 1257 keeps coming back

James Edmonds 811

Hi,

I have two failover clusters created, for which I did NOT pre-create the DNS records for the cluster or role names.
The DNS records are created when the cluster/role is brought online.

On one of the clusters, I keep getting event ID 1257, where it fails to register or update the DNS entry for the role running on the cluster.
If I delete the existing record, and restart the role, it creates successfully.

I am trying to understand why, if the cluster creates the record, this error keeps coming back?
What can I do to prevent this from constantly complaining about this, when both cluster nodes have access to that DNS record?

Thanks
James

James Edmonds 811 Reputation points

2022-06-15T10:16:32.077+00:00

My ocurrence of this issue just started again this morning.
I am able to resolve by taking the clustered role name resource offline and onlining it again.

What I have noticed, is that for a clustered role on another cluster, the permissions on the DNS entry for the role name resource lists the cluster name resource as having permissions.
On the one that was having issues this morning, it shows the role name resource as having permissions on itself, rather than the cluster name resource.

Should, when onlining the role name resource,, the cluster name object be what is given permissions on the DNS entry, rather than the role name object?

Banging my head against a brick wall here and started to get a bit frustrated.

Cheers
James
Jack Dobiash 6 Reputation points

2022-06-15T17:54:43.587+00:00

Hey James, I checked my cluster objects and at least in our version (Server 2016), the actual Cluster itself 'owns' and updates the DNS objects on itself and on any roles that it is hosting. If you check the 'Advanced' security area of the DNS record, it should show you who owns it as well. We haven't had any issues with the individual roles themselves (so far), just the main CNO object is what isn't updating it's DNS record.

I did notice that the pwdLastSet timestamp of my CNO is different than the individual roles (although they are within a day or so of each other), meaning they don't all get updated at exactly the same time. Did the password on that role object recently get updated? Just curious if your issue is similar to mine.

Lastly, if you check the Diagnostic Failover Cluster event logs, you might be able to see something that occurs at the time when the DNS renewal is attempting. Those logs are pretty bloated, however, so it can be a bit of a pain to find stuff (try and filter out 'information' events), and usually they rollover pretty quickly (if the log size isn't increased) so you almost need to find it while the event is occurring.

Hope this helps!
James Edmonds 811 Reputation points

2022-06-16T10:59:24.99+00:00

Hi Jack,

I'm interested in what roles you are hosting?
I think my issue is that, for both my CNOs and my SQL role DNS name resource, the owners of the DNS objects are the cluster name objects themselves.
In the case of my problematic file server role, the owner is the role itself.

See below. Top two entries are the CNOs. Bottom left is the problematic file server role, and bottom right is SQL role:

It seems as though when the file server role DNS name resource is brought online, it is not setting ownership of the DNS entry correctly?

I am not sure if my password reset times suggest my issue is the same as yours?

Cheers
James
Jack Dobiash 6 Reputation points

2022-06-28T00:24:44.247+00:00

So aside from Hyper-V, the two 'Roles' we have on our non-HyperV cluster is DHCP and the 'Generic' one running Certificate Services. No issues with either of those. Our main CNO Name issue came back today on our Hyper-V cluster, right on schedule (22 days between password changes).

I just got done doing some more testing and the results are 'interesting' to say the least. Instead of just stopping and starting the 'Name' role to fix it again, I decided to actually delete the DNS record right before it was about to attempt to renew it. To my surprise, it registered the new record in DNS, but really to my surprise, the 'owner' of the record was no longer the CNO, but one of the 'Roles' on the Cluster, the Hyper-V Replica Role, to be precise. Keep in mind this is the DNS record for the actual CNO, not the role, but yet it used the Replica Role account to register the DNS name. The Replica role has it's own DNS record, which is owned by the main CNO, and it's been updating fine this whole time.

As another test, I've now 'stop/started' the 'Name' role on the cluster, and I'm going to see if, when it updates the DNS record in ~24 hours, it fails again. I suspect it will, as then it will be trying to use the proper Cluster account name again and it won't have access anymore.

It'll be interesting to see what happens when I try this on my other non-Hyper-V cluster, once it starts having the problem (which should be here in the next day or so).
James Edmonds 811 Reputation points

2022-06-28T09:04:04.817+00:00

I think this too is what we are seeing on one of our clusters.
The owner of the DNS record after deleting it manually, and having the cluster recreate it by bringing it offline and online, is one of the roles rather than the cluster itself.

If you're environment then behaves like mine, it'll be fine for ages, then at some yet to be determined interval, it will start triggering event ID 1257, saying it cannot be updated.

I'm still monitoring ours, but due to a number of power outages recently, our clusters have been fully stopped a number of times, so I have no idea what the real interval between event ID 1257s is at the moment.

Let me know how yours goes.

Cheers
James
Jack Dobiash 6 Reputation points

2022-06-30T02:03:21.46+00:00

So our other Cluster was due for the problem as well, so I waited until it started throwing out event ID 1257 again, then just deleted the DNS record for that clusters main CNO. Similar to before, once it tried again (it tries every 15 mins once it fails) it registered in DNS, but now the "DHCP" role owns it. It's almost like I'm having the exact opposite issue as you, where my main CNO DNS object is now being 'owned' by the roles it maintains, rather than by itself. So far, no errors have ever happened when attempting to register the roles DNS, but it might be that I'm not waiting long enough after the main CNO quits being able to update itself.

If I delete the DNS record, and then stop/start the 'Name' role on the cluster itself, it re-registers the DNS record under it's own name properly. The next time the password is updated is when it switches to one of the roles.

I'm still leaning towards the thought I had when I originally posted, which was that my cluster basically 'forgets' how to login under it's own name after it changes the password on the CNO, but somehow it's falling over to using one of the roles credentials. I suppose I can just add both the roles and the main CNO object as being able to write to the DNS record. This will probably make it work, however I would still consider this a 'workaround' to a bug that was introduced within the last 6 months or so.
James Edmonds 811 Reputation points

2022-07-01T18:46:31.437+00:00

Yes, to me it feels like a bug, as the cluster CNO should take ownership of its own DNS record.
I can see no logical reason it wouldn't do that, unless there was a bug that meant it didn't try, or skipped to using the role CNO if the cluster is not able to for some reason.

Very odd.

I cannot raise with Microsoft sadly, as we have no premier support, but perhaps if enough people notice the issue and post here, they may look into it.

Cheers
James

15 answers

AaronB 16 Reputation points

2022-11-03T17:14:53.383+00:00

I was seeing a lot of 1257 errors in our file cluster when we added 3 new nodes. We use static DNS records for pretty much everything that isn't a user endpoint, so I don't need the servers to update anything in DNS. I was able to eliminate the 1257 errors by going in to the NICs on the nodes showing errors, under IPV4 Advanced DNS options, uncheck "Register this connection's addresses in DNS"
Please sign in to rate this answer.

3 people found this answer helpful.

0 comments No comments
Sign in to comment
Limitless Technology 43,946 Reputation points

2022-03-03T11:52:53.03+00:00
Hi @James Edmonds

Thank you for your question and reaching out.

I can understand you are facing event 1257 ID of DNS.

Please follow below steps in order to resolve the issue.

Please delete the CNO ‘A’ record from DNS console.

Add the same record and verify that “Allow any authenticated user to update DNS record with the same owner name” option is selected.

Hope this answers your question :)

--
--If the reply is helpful, please Upvote and Accept as answer--
Please sign in to rate this answer.

1 person found this answer helpful.
James Edmonds 811 Reputation points

2022-03-03T13:32:11.81+00:00

Hi Limitless,

I can do this, but I was under the impression that when the cluster/role creates the DNS entry, it would apply the appropriate permissions on the DNS entry for it to be able to manage it itself?
I can see the records have the cluster computer names and role computer names added having read/write privileges, so I was hoping that would be all that is needed?

Is it not expected behaviour for the cluster to create the records in a way they can manage them themselves?

Many thanks.

James Edmonds 811 Reputation points

2022-03-14T09:29:04.223+00:00

After the last time I deleted the entry and had the cluster recreate it, the error went away for a while. It's kicked in again today.
Checking the cluster name DNS record, the cluster has read and write permissions on the record:

Is this not enough for the cluster nodes to be able to update the record?
If not, why is it not setting the appropriate permissions when it creates the entry in the first place?

Cheers
James
Sign in to comment
Joe Bruns 21 Reputation points

2022-07-22T14:46:44.317+00:00

As of 7/22

MS has identified it is indeed an issue on their end. Due to the nature of the 2016 maintenance, they have to examine code from November of 21 to April of 22 to zero in on the culprit and kill the bug.
Please sign in to rate this answer.

1 person found this answer helpful.
James Edmonds 811 Reputation points

2022-07-22T15:42:51.487+00:00

That's good....in a way :)

If you want to let them know I, as a Server 2022 user, see the same issue, I will gladly speak to them if they want someone with an impacted 2022 environment for diagnosis/testing etc.

Thanks
James

Joe Bruns 21 Reputation points

2022-07-22T16:11:14.017+00:00

I have let them know it affects 2022 as well.

James Edmonds 811 Reputation points

2022-08-15T10:25:36.627+00:00

Hi Joe,

Have MS provided you with any update on this issue as of yet?

Cheers
James

Joe Bruns 21 Reputation points

2022-08-15T13:10:17.967+00:00

Trying to reach back to "Jason S" who worked the case. I don't have his email since MS masks that.

jaspiers 1 Reputation point

2022-08-17T21:39:19.363+00:00

Hi Joe,
Sorry for the delay in response. There are currently two different issues being tracked by Microsoft which result in the event ID 1257. In Server 2016, we are seeing the 1257 error occurring after password resets. There was an initial fix for this released in summer 2021, and an update to it released in November 2021, which was enabled in the March 2022 release. One of the updates added the ability to use the Virtual Computer Object (VCO), i.e. the server name used for individual resources, as well as the Cluster Name Object (CNO), as an authenticating resource in order to update DNS entries. The behavior we would see here, was if the CNO was taken offline/online, either manually or by failover, the issue would cease, until the next CNO password change. If this behavior was noted, then workarounds such as deleting and recreating the DNS object, with the checkbox to allow any authenticated account to update it, would not fix it long term.

At the "nuts and bolts" level, what was observed was that the system would try first to use the VCO to update the DNS, which would fail, and log the 1257 error. The system would then try the CNO to update the DNS, which would succeed without further notification to the system.

The issue being seen in 2022 is a different issue, which I have no current details to share. You would need to open a support ticket on that one as it is still under investigation.

Thanks!

James Edmonds 811 Reputation points

2022-08-18T09:56:12.43+00:00

Hi,

I have still not been able to confirm if our issue in 2022 happens at the time of password resets, but it does seem to be ocurring with some level of regularity, which could indicate that it is indeed happening because of the same reason as Joe's issue in 2016 (password reset time).
The behaviour we see is basically the same, other than the fix you mention saying it tries the VCO and CNO, then stops logging errors once successful. The only difference for us, is we normally see a VCO from a resource in the cluster listed on the security of the DNS record, when we would expect to see the CNO, but it still fails to update the record. We then continue to receive event 1257 every few minutes or so.

To me, it feels like the same issue, and we cannot open a Microsoft ticket as we do not have any support agreements with them, hence posting here in the forums.

If there is an issue with 2022 that is similar to 2016, I shouldn't have to pay to open a ticket to bring it to Microsoft's attention and to fix it.
Is there anything we can do to confirm if this is the same issue as in 2016?

Cheers
James
Sign in to comment
Sivert Solem 16 Reputation points

2022-08-15T13:37:41.677+00:00

We have also experienced this issue.
In our case we have noticed that the cluster attempts to update DNS records using the credentials of a random VCO in the cluster.
A workaround is to put all VCO's in a security group and give the group permission to update DNS entries for all roles in the cluster.
Please sign in to rate this answer.

1 person found this answer helpful.

0 comments No comments
Sign in to comment
Robert Berndt 6 Reputation points

2022-08-30T12:26:33.547+00:00

I've seen this on any of our 2016 or newer clusters. The only way I've been able to have it go away is to go into the cluster name in DNS, make sure that the cluster name has "Full Control" of that DNS entry. Then the errors go away.
Please sign in to rate this answer.

1 person found this answer helpful.

0 comments No comments
Sign in to comment