From my understanding, you have today all the objects synchronized from on-premises AD to Azure AD/Exchange Online (EXO) and you want to start managing distros, contacts, mailboxes from EXO only (i.e. use cloud-mastered objects).
If the mailboxes are still in on-premises Exchange, then first you need to move all the mailboxes to EXO.
After having all the mailboxes in EXO, you'll have to transfer the source of authority of all the objects in Azure AD by stopping AAD Connect synchronization and disabling DirSync on the tenant. This is a process that can take many days depending on the size of the tenant as it will convert every single DirSyncEnabled object to a cloud-only object mastered in the cloud.
If you want to keep managing users from on-prem, you can reconfigure AADConnect with a sync scope (e.g. OU filtering) to only synchronize these users (and not the other object like distros and contacts) and re-enable synchronization on the tenant. After this, AAD Connect will "re-match" the existing users in Azure AD with the users from local AD and convert these objects back again to DirSyncEnabled objects in Azure AD. For these "hybrid exchange" objects though, you might still need to manage Exchange properties from on-premises so I would recommend keeping one last on-prem Exchange Server for management purposes only.
For more information, please read: https://learn.microsoft.com/en-us/exchange/decommission-on-premises-exchange