The licensing requirement depends on how you want to do the renewal. For SSPR password writeback you do need the Premium P1 license. But if the users just change their passwords rather than reset them (not using SSPR but via Office 365 portal, My Apps, or the Windows 10 sign in page), you can just enable the password writeback option in Azure AD Connect.
If you want to have cloud-only Azure AD accounts then yes, you will need to remove the on-prem accounts re-create them in Azure. There are some limitations though to the cloud-only approach since it's not a full replacement for an on-premises Active Directory.
https://learn.microsoft.com/en-us/microsoft-365/enterprise/cloud-only-identities?view=o365-worldwide
https://learn.microsoft.com/en-us/azure/active-directory-domain-services/scenarios