Since I cannot repro, I would need traces or logs of some sort to investigate further...
AD FS uses the SNI extenstion of TLS so it is critical that the name that shows in the output of Get-ADFSProperties
is matching what the user-agent is using (what the user typed in the browser).
Maybe the user's browser has a proxy configured? Or other network conditions (or other browser's settings).