Hi,
Usually I have faced similar issue and the issue was with some users having special permissions or blocked inheritance due to some admins blocking top level permissions, please check the the account specified in Azure AD Connect must have the appropriate permissions and options set on that specific OU and users security tab.
Hope this helps.
JS
==
Please accept as answer and do a Thumbs-up to upvote this response if you are satisfied with the community help. Your upvote will be beneficial for the community users facing similar issues.