HI
"So the question I have is, is it worth getting graphic cards for the new servers?
if your have enough memory on hyper-v host physical server and there is no need to use 3D rendering for your guest vm ,i think you need not to purchase a Nvideia Quadro P4000 graphic card for your new server.
Plan for GPU acceleration in Windows Server
https://learn.microsoft.com/en-us/windows-server/virtualization/hyper-v/plan/plan-for-gpu-acceleration-in-windows-server