Hello Julien,
Thanks for your reply with patience.
If you prefer the Pass Through Authentication (PTA) to sync your AD users to Office 365 Azure AD, this is OK. However, you would not be able to continue using ADFS integrate your own SSO service for user authentication. For more details about Pass Through Authentication with AAD conect, please refer to User sign-in with Azure Active Directory Pass-through Authentication.
Moreover, if you still want to use AFDS for user authentication, you cannot enable Pass Through Authenticaiton, and for the external users authentication, you need to deploy ADFS proxy just as we talked before, thanks. And if you don't use ADFS in the future, and only want to sync your AD users to Office 365 Azure AD with SSO experience, you could try to enable the Semless SSO feature with AAD connect ( Not your own SSO) and in this way, your Office 365 users would get a SSO experience from the domain-joined devices. For your reference Azure Active Directory Seamless Single Sign-On.
Please feel free to let me know if you have any other concerns, thanks.
Regards,
Oliver