Replace DC server (server 2019)

FotS 81 Reputation points
2021-09-12T06:20:50.94+00:00

We were hit by a ransomware attack. Because the company hadn't invested any money into offsite backups, we only had onsite and, yeah, for all intents and purposes they are toast right now. The 2 DCs we had were fine, until someone powered on an infected machine I had powered down. Now both DC servers are at least partially encrypted. I don't see signs of the virus itself on them (yet), but I don't want to take any chances. I'm actually afraid to logout or even disconnect from DC01 for fear I won't be able to log back into it (I can't log into DC02 anymore due to the encryption).

I've powered down DC02 and using our virtual environment, spun up another server (DC03). I've added the same roles to it as what DC01 had (AD DS, DHCP, DNS, and NPAS), promoted DC03 as a DC in AD and started some work on DNS (on DC01 I've added DC03 as a Name Server), but when following guides for steps that continue to the new server, I've noticed all the zones are already visible on DC03, so I'm not sure what to do.

I really don't have the level of expertise to do all of this, but the company lacks the money to hire outside help. I've found some guides for complete replacements of DCs, but they're old (Server 2008 or older), and I'm not sure how much has changed over the versions (sorry, should mention that all of our servers are 2019 and I know I've already run across some guides saying that even going from server 2008 to 2016 require some additional steps like upgrading to 2012, first). Frankly, I don't even know what things I should be asking/looking out for. Until I was looking at some of these guides, I had never even heard of FSMO in relation to AD.

I need to get the AD, DHCP, DNS, and NPAS services transferred over to DC03, plus whatever else in there that I may not know about. NPAS I think(?) is being used for a RADIUS connection from the site's internet firewall for VPN. That said, there's some software installed on DC01 for the firewall, too, for the purposes of the web filters, so I think I'm going to engage the firewall support people to move the software and get VPN going through that instead. There's also Azure AD Connect software running on DC01, too (primarily used to sync our AD accounts with our Office 365 email accounts). That should be it.

Once all of that is transferred over, I want to shut DC01 down, then either rename and re-IP DC03 to match DC01, or spin up another DC server to match DC01's name and IP (would that be easier?). I know there have to be a lot of references and pointers to its name and IP, so I want to make sure at least something exists with it's identical info. If it makes it any easier, I think we have like less than a half dozen PCs that are still functional at this point, anyway. XD

What a nightmare this is turning into.... I need help. :(

Windows Server 2019
Windows Server 2019
A Microsoft server operating system that supports enterprise-level management updated to data storage.
3,444 questions
Active Directory
Active Directory
A set of directory-based technologies included in Windows Server.
5,822 questions
0 comments
Accepted answer
  1. Dave Patrick 426K Reputation points MVP
    2021-09-12T12:38:56.58+00:00

    For the additional domain controllers I'd use dcdiag / repadmin tools to verify health correcting all errors found before starting any operations. Then stand up the new 2019, patch it fully, license it, join existing domain, add active directory domain services, promote it also making it a GC (recommended), transfer FSMO roles over (optional), transfer pdc emulator role (optional), use dcdiag / repadmin tools to again verify health, when all is good you can decommission / demote old one.

    For DHCP do you have a backup or starting over?

    For the NPAS question I'd start a new thread over here.
    https://learn.microsoft.com/en-us/answers/topics/windows-network-access-protection.html

    Do not install the vpn on a domain controller. I'd stand up a separate instance for this purpose.

    --please don't forget to upvote and Accept as answer if the reply is helpful--

    1 person found this answer helpful.
    0 comments

8 additional answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Dave Patrick 426K Reputation points MVP
    2021-09-12T13:10:52.547+00:00

    all the errors being created were related to not being able to contact DC02

    You'll need to perform some cleanup to remove the failed ones.
    https://learn.microsoft.com/en-us/windows-server/identity/ad-ds/deploy/ad-ds-metadata-cleanup
    https://techcommunity.microsoft.com/t5/itops-talk-blog/step-by-step-manually-removing-a-domain-controller-server/ba-p/280564

    As for VPN/web filter, the firewall tech support people actually say their software works best directly from a DC server?

    Never install the vpn role on a domain controller. The multi-homing will always cause no end to grief for active directory DNS

    --please don't forget to upvote and Accept as answer if the reply is helpful--

    1 person found this answer helpful.
  2. Dave Patrick 426K Reputation points MVP
    2021-09-12T17:31:48.043+00:00

    Looks like replication is badly broken. Not sure of the steps history here but it may be some new domain controllers were added without confirming domain health further complicating issues.

    One option may be to try a non authoritative sync on the broken ones. (check the event logs for details)
    https://support.microsoft.com/en-us/help/2218556/how-to-force-an-authoritative-and-non-authoritative-synchronization-fo

    For DC01 I'd check that it isn't somehow firewalled off, also check the SRV records are there
    https://learn.microsoft.com/en-US/troubleshoot/windows-server/networking/verify-srv-dns-records-have-been-created

    --please don't forget to upvote and Accept as answer if the reply is helpful--

    1 person found this answer helpful.
  3. FotS 81 Reputation points
    2021-09-12T13:06:16.247+00:00

    @Dave Patrick , it's not letting me post a comment to your answer for some reason, so I'm posting my comment as an "answer".

    I appreciate the response, though I've been powering ahead this whole time (didn't have a choice). XD I did run the dcdiag commands, though, and all the errors being created were related to not being able to contact DC02 while I had it powered down. Powering it up seemed to correct everything. However, I had to manually remove it from the domain as I was unable to actually log into the server anymore.

    DNS was actually easier than the guides I was finding led me to believe. The person who initially set all of this up apparently used the option to sync it with AD, so merely adding the server name into DC01's DNS settings was enough for it to replicate over to DC03.

    I also got the FSMO roles moved over to DC03, though schema fought me some (I eventually got it).

    DHCP I was able to pull a good copy off, however it's not working. I suspect an issue with the helper IP configs on the core network switches for that, though....

    At this point, I've successfully fully removed DC02, then created a new DC02 to take its place and have it setup with AD DS, DHCP, and DNS, promoted as a DC. DC01 is now also fully removed from the domain and I'm in the process of setting up a new replacement for that. Hopefully once I get DHCP moved back over to that one, DHCP related services will start working again. XD

    Oh, and GC seemed to take care of itself. Each server as I set it up with AD DS and promoted it as a DC automatically got that checked off.

    I think I got lucky....

    As for VPN/web filter, the firewall tech support people actually say their software works best directly from a DC server?

    0 comments
  4. Dave Patrick 426K Reputation points MVP
    2021-09-12T13:37:44.98+00:00

    You're welcome, sounds good then.

    --please don't forget to upvote and Accept as answer if the reply is helpful--

    0 comments