Hi Senne Colson,
I hope you used the same VM size and security type (confidential VM) for the Ubuntu VM as you did for the Windows VM, based on the specifications shared above.
Could you please try launching a VM with Windows Server 2022 Core and install the NVIDIA GPU Driver Extension for Windows. https://learn.microsoft.com/en-us/azure/virtual-machines/extensions/hpccompute-gpu-windows#operating-system-support
If you have any queries, please do let us know, we will help you.