You can create Windows Server Virtual Machine in Azure without having any licenses.
Windows Server licensing costs are already include into VM monthly cost
To deploy Windows 10 on Azure users must have one of the below subscription licenses in order to use Windows 10 images in Azure.
Eligible subscription licenses:
Microsoft 365 E3/E5
Microsoft 365 F3
Microsoft 365 A3/A5
Windows 10 Enterprise E3/E5
Windows 10 Education A3/A5
Windows VDA E3/E5
How to create Windows VM in Azure
https://learn.microsoft.com/en-us/azure/virtual-machines/windows/quick-create-portal