Hi,
The lowest level you can assign policy is at resource group level. I would suggest to divide the VMs to different resource groups depending on what policies you want to assign them. The purpose of a resource group is to manage the resources in it together. A nasty workaround that I do not recommend is to assign the same policy to the same resource group multiple times but each policy to exclude the VMs for which do not apply. That is not good approach as every time you add new VM you will have to update those exclusions.
Please "Accept the answer" if the information helped you. This will help us and others in the community as well.