If you are running the cmdlet in the context of a given user, both the user permission and the permissions granted to the Graph SDK for PowerShell play a role. To check the latter, use the Get-MgContext cmdlet, then compare the output of the Scopes property against the permission requirements for the cmdlet/Graph endpoints you want to use.
For example, a call against /users/{userId}/calendar endpoint requires at least the Calendars.ReadBasic permission (assuming you only want to check your own calendar). To make sure the cmdlet will be run with such permissions, run:
Connect-MgGraph -Scopes Calendars.ReadBasic