I did a workaround by configuring IEMode to Edge and added only those Infopath forms library addresses to the Enterprise Mode Site list and set those addresses to work in IE10 emulation.
So now everything else works as it should inside Edge, but when user navigates to Infopath forms library that has non-browser enabled forms that site/library loads in IEMode (IE10) and user can open those Infopath-forms with Infopath client application without problems.
What are the odds that when I just got this working, there's now fix to this problem in the latest SP2019 October 2021 CU which was released yesterday.
- Fixes an issue in which the client application can't be started to open a non-browser-enabled InfoPath form in modern browsers. Additionally, you receive the following error message:
"This form cannot be opened in a Web browser. To open this form, use Microsoft InfoPath."
I will try to remember to update this post when I have tested that October 2021 CU.