Hi,
Based on my knowledge, since the user accounts are created in Active directory and synchronized with the Office 365 using AD Connect, Exchange server needs to be deployed for managing the mail attributes for those users and cannot be managed from Office 365. This is not because of Exchange hybrid but with the Active directory synchronization.
So, for creating the mail users, deploy the Exchange server and import the CSV because New-MailUser is an exchange command-let and requires exchange powershell.
May be you can install the Exchange server with trial license which will run for 180 days. You can complete the migration by that time and configure the Exchange Hybrid with Hybrid license in future. The reason for Exchange hybrid in future is, until the Active directory is synchronized with office 365, you need to have at least one exchange server on-premise for management of mail attributes.
https://learn.microsoft.com/en-us/exchange/decommission-on-premises-exchange