@LoudCloudDragon_caseyDB Apologies for the delay in response and all the inconvenience caused because of the issue.
Can you please confirm if you got a chance to go through below documentation:
https://learn.microsoft.com/en-us/azure/virtual-machines/extensions/hpccompute-amd-gpu-windows
https://learn.microsoft.com/en-us/azure/virtual-machines/windows/n-series-amd-driver-setup
The above mentioned are the official documentation from Microsoft side and are recommended while installing AMD GPU on Azure.
Based on which NV4 series you have installed the NVv4 Virtual Machines support up to 32 vCPUs, 112GB of RAM, and 16 GB of GPU memory max. Some info as below:
Hope it helps!!!
Please "Accept as Answer" if it helped so it can help others in community looking for help on similar topics.