Thank you for reaching out to Microsoft Q & A Platform
The number of VM's per resource group parameter is fixed and cant be changed :
You can try out creating Azure policy for the second use case of only creating certain types of VM's in a resource group .
Request you to please accept the answer if this was helpful in implementing your use case .
Thanks & Regards,