Well, if you want to use AADConnect, that is the only supported config.
If you want to use AADConnect and Azure AD Cloud Sync, then you can do what you want.
In other words, one forest has AADConnect, the others have Azure AD Cloud Sync
https://learn.microsoft.com/en-us/azure/active-directory/hybrid/cloud-sync/what-is-cloud-sync