Hello,
The most likely cause for this, is that Time synchronization depends on the name resolution of the Time Server or NTP configured in the system. This name resolution may be only available for IPv4 records (known as "A" records in Active Directory DNS) while IPv6 interfaces will need to have a DNS record that resolves IPv6 records (known as "AAAA" records in AD DNS) for the Time Source. Be aware that some public NTP Servers will not provide IPv6 or AAAA record resolution.
--If the reply is helpful, please Upvote and Accept as answer--