Hi @Noah Venema ,
In the requirements page for ADConnect V2, is specified that for the installation of Azure AD Connect server, Windows must have a full GUI installed. Installing Azure AD Connect on Windows Server Core isn't supported. I don't know if this applies to upgrading from a previous version.
how-to-connect-install-prerequisites
I Hope this helps.
==
Please "Accept the answer" if the information helped you. This will help us and others in the community as well.