Not sure if you have check Cross Tenant Sync Preview feature in Azure AD External Identities, I will suggest you to review this for collaborations. https://learn.microsoft.com/en-us/azure/active-directory/multi-tenant-organizations/overview
Btw please share your config for this B2B setup, so I can replicate the issue.
Hope this helps.
Please Accept the answer if the information helped you. This will help us and others in the community as well.