Similar to Siva's reply here, you will want to navigate to
C:\Users<your username here>\AppData\Local\Microsoft\Windows\FileHistory
Now, if FileHistory had been turned on before, using the old username, it will have a configuration folder, and the Config1.xml, and Config2.xml files will contain many tags using the old PC name. Instead of wiping these files out, open each file in Notepad and search for your old PC name and replace with your new name, remember to save.
Also, if you want to keep a backup that was already created with the old PC name, you will need to find it, rename the main folder ("old PC name"), and within that folder, rename the tags in those config files as well (as you did in the local config files).
When I tried deleting my local config files, it would recreate them using the backup's config files, which were also incorrect, so I just manually renamed and mine is running nicely now under the new PC name.