Yes, various parts of SharePoint Online will use IPv6 for services configured to do so on Microsoft's side. There is nothing for you to adjust as you don't control that level of infrastructure.
You can see the endpoints listed at https://learn.microsoft.com/en-us/microsoft-365/enterprise/urls-and-ip-address-ranges?view=o365-worldwide.