It seems this is a bit of a dinosaur that takes us right back to the numerous .ini files that existed under Windows 3.1. The Microsoft engineers might have forgotten to include a suitable management tool for such connections.
VPN and dial-up details are kept here:
C:\Users%UserName%\AppData\Roaming\Microsoft\Network\Connections\rasphone.pbk
To delete an existing connection, use notepad to open the above file, find the unwanted name, then find the next square bracket [, then delete everything in between. You have to log off to make the change stick.
_________________________
If this response resolves your problem, please click "Propose as answer" so that the thread can be closed. Thanks.