Are you sending a delete or cancel?
Delete is needed here according to the documentation:
https://learn.microsoft.com/en-us/graph/api/event-delete?view=graph-rest-1.0&tabs=http
Cancel only works for the organizer:
https://learn.microsoft.com/en-us/graph/api/event-cancel?view=graph-rest-1.0&tabs=http