@Adrian& Thank you for reaching out to us, as per my research and findings, prior to changing the domain Authentication type, FederationConfiguration must be created in order to switch from Managed to Federated.
Command to create DomainFederationConfiguration: https://learn.microsoft.com/en-us/powershell/module/microsoft.graph.identity.directorymanagement/new-mgdomainfederationconfiguration?view=graph-powershell-1.0
Let me know if you have any further questions, feel free to post back.
Please remember to "Accept Answer" if answer helped, so that others in the community facing similar issues can easily find the solution.