I found the solution and I have wasted more than a day on this. I really wish Azure team can provide correct versions bundled with DSVMs
The NVIDIA library that is bundled along with Ubuntu 18.04 Gen 1 DSVM is Nvidia-495. This is not supported by Ubuntu 18.04. I had to try various installs, refreshes, network settings etc. to arrive at this junction. Finally I found in syslog that Ubuntu 18.04 was ignoring library 495 and hence GPU is not loaded.
Then I had to do a lot of circus to remove 495 in a clean manner and install 470 which is supported by Ubuntu 18.04 and then it worked.
Meanwhile, the DSVM page in Microsoft also mentions that K80 (NC6) machines are loaded with 470 drivers. However in practice I found that it comes with 495 which was the root cause of this issue.
Can someone from Microsoft Azure team, please update your image for DSVM such that it comes preloaded with NVIDIA driver 470 instead of 495?