Finally got it to work!
Uninstalled all OLE DB versions installed on the PC - rebooted
Attempted another install of SSMS 19 - no dice, still gave the error.
Installed OLE DB 18.6.5 - it mentioned in the install that an older version was installed and that it would upgrade it. (I had uninstalled all versions found but hey registry keys and all that)
Then installed SSMS 19 and it installed OK!
My conclusion is there is some magical pixie dust in OLE DB 18.6.5....
as I had already ran SFC /scannow DISM cleanup-image etc...