@gsaunders You are right on with your research, Azure presents you with virtual 'Cores', but they appear as physical cores to your VM. Assume that it is 1 Azure core = 1/2 of a dual core physical processor with the listed speed.
If your SQL server needs 4 dual core processors, an 8 core VM is what you want. Your OS should see it as 8 distinct processors.
One thing to keep in mind in Azure is that changing your VM size is easy, and only requires that you restart your VM (in most situations, there are some cases where your VM will not need to reboot). If you see that you need more or fewer cores, it is easy to change the size at any time.
Make sure to read the performance best practices for SQL Server on Azure VM before setting up your server!