It's actually a minimum of 16-cores required per customer with 8-cores assigned per virtual machine. So yes, you always need to purchase at least 16-cores and in your case it sounds like you are going to therefore need 2 x 8-core 1 year CSP subscriptions for Windows Server 2022 Standard.
You will use up 8 cores on your 4vCPU VM but the good news is that you will still have another 8 cores available to license any other or future VMs that you deploy up to 8vCPU in size.
https://learn.microsoft.com/en-us/azure/virtual-machines/windows/hybrid-use-benefit-licensing