If you have sufficient permissions, try running the following cmdlet to reset the delegate collection:
Remove-MailboxFolderPermission mailbox:\Calendar -ResetDelegateUserCollection
More information about the switch here: https://learn.microsoft.com/en-us/powershell/module/exchange/remove-mailboxfolderpermission?view=exchange-ps