Is there something overriding the setting?
This setting may be overridden and become read-only if EnableMIPLabels is set to True and a guest policy is associated with the sensitivity label assigned to the group. If the AllowToAddGuests setting is set to False at the tenant level, any AllowToAddGuests setting at the group level is ignored. If you want to enable guest access for only a few groups, you must set AllowToAddGuests to be true at the tenant level, and then selectively disable it for specific groups.
https://learn.microsoft.com/en-us/azure/active-directory/users-groups-roles/groups-settings-cmdlets