If creating a VM using PowerShell Az module, there is -DisableBginfoExtension switch to skip installing Bginfo extension, so you save time it takes to install it and don't need to uninstall.
New-AzVM -VM $VMConfig -Location $Location -ResourceGroupName $ResourceGroupName -DisableBginfoExtension