Security groups by default can only be managed by admins. For other group types, you have the relevant controls to prevent creation and membership changes.
If you want to further restrict admin access, consider using Administrative units, and take a look at the role-assignable groups functionality.
You can add more details on the planned use cases and group type needed, if the above does not suit your needs.