Hello Marcel Nguyen,
Thank you for posting your query in the Microsoft Q&A Community.
Aside from the group not being a dynamic group and users trying to leave the group aren't Owners, the group must not be a synced group either from Exchange or On-prem.
If all these are not the case, I would advise you to create a ticket for the Support Team to have a closer look at the issue.
Let me know if further assistance is required.
Babafemi