Hi Leanne L TT,
From your problem description, you would like to view different folders from group’s document library under attachment file. At this moment, user can only view the one site folders by default. However, as the workaround, you can switch among the different group’s libraries by selecting the tab above to go back to the home page.
If you would like to visit the library from different sites, you can change the SharePoint site address with the permission from address bar to view the content.
Sorry for the inconvenience caused to your works. We suggest you can vote this great idea in Outlook UserVoice. It is the most efficient way to let product team know about this limitation. A lot of users have posted their great ideas on UserVoice platform. It helps Microsoft products and services to make them better. Thank you for your understanding.
Best regards,
Dihao