Assuming you have a CSV file with fields designating the user (UserUPN) and Manager (ManagerUPN), you can do something like this:
Import-CSV blabla.csv | % { Set-MgUserManagerByRef -UserId $UserUPN -OdataId "https://graph.microsoft.com/v1.0/users/$ManagerUPN" }
where we are using the Graph SDK for PowerShell.