You dont need a license to sync on-prem to Azure in most cases:
If using AADConnect, you would need one for more than 500,000 objects. That may also apply for cloud sync, but its not spelled out
If you want to leverage additional capabilities in Azure, then a P1 or P2 is required:
https://www.microsoft.com/en-us/security/business/microsoft-entra-pricing