I don't know how to do it manually. But I think I have found two solutions.
One is a command line tool: https://www.codeproject.com/Articles/37133/Simple-Version-Resource-Tool-for-Windows
The other other is to use an .rc resource file.
Since my needs are simple, the .rc approach will work for me.
In visual studio I right click the project and Add/Resource.
Select Version from the list of Types, and then edit it to my preferences.
I create two resource entities this way for one project, one for Release and one for Debug.
I right click on those and edit their Properties/Exclude File From Build, to be excluded from the builds that don't apply.
So far this seems to work very nicely.