I'm starting to get somewhere identifying the cause, but sadly not the fix.
First, I've given up on Microsoft.Speech. I get nothing but silence no matter what I do.
I can alter the behaviour of System.Speech depending on the installation package I use. It seems that the installer that fails is missing the inclusion of a DLL.
If I use the Click Once function within Visual Studio, it works. Unfortunately, it does not put the program in Program Files or Program Files (x86) where it belongs. If I use the Microsoft Installer Project extension, the program is installed in Program Files (x86) but fails when creating the System.Speech.SpeechSynthesizer object.
I compared the files produced by the installation files of the version that worked and the version that didn't and noticed two different versions of System.Speech.dll in the one that worked. They have almost identical timestamps (23 Oct 2021 at 10:48 and 10:49) but one is more than double the size of the other (648K vs 301K). I opened the 648K version and viewed the version.txt file within it to see "Internal implementation package not meant for direct consumption. Please do not reference directly. Provides types to perform speech synthesis and speech recognition."
It does appear that in spite of having the same name, both are required. Could it be that the Microsoft Installer Project extension is not including this second and larger file?