Hello Jocelyn Dubois,
Thank you for contacting Microsoft Support!
As you mentioned, your token has Calendars.ReadWrite
but not Calendars.Read.Shared
. This means you should only be able to access your own calendars and not shared ones.
I have followed the same steps provided by you and not getting calendars of other users in the Graph API response.
I would recommend you raise a support case with Microsoft Graph, a Support Engineer will be able to look into this issue and assist you better. You can raise support ticket from New support request - Microsoft Entra admin center or https://admin.microsoft.com/#/support/requests.
Hope this helps.
If the answer is helpful, please click Accept Answer and kindly upvote it. If you have any further questions about this answer, please click Comment.