I had opened a ticket with Microsoft on this specific issue. Response from support is below.
This is indeed a limitation with the TPM. Specifically, we have seen this behaviour with TPM v2.0 revision 1.16 (higher revisions do not exhibit this issue).
The TLS 1.3 RFC requires the RSA-PSS signature algorithm salt to be equal to the length of the output of the digest algorithm (also applies to TLS 1.2).
On the affected machines, the salt size does not match:
[TPM version: PCP_PLATFORM_TYPE: "TPM-Version:2.0 -Level:0-Revision:1.16-VendorID:'IFX '-Firmware:458815.858368"]11/15/22-09:38:07.1611301 [Microsoft.Tpm.DebugTracing.KSP] [FunctionEnd] PartA_PrivTags=16777216, Name=DetectPSSPaddingSalt, HResult=The operation completed successfully. (0x00000000)
11/15/22-09:38:07.1611392 [Microsoft.Tpm.DebugTracing.KSP] [FunctionEnd] PartA_PrivTags=16777216, Name=TpmKey20Rsa::SignHash, HResult=The requested salt size for signing with RSAPSS does not match what the TPM uses. (0x40290423)
11/15/22-09:38:07.1611529 [Microsoft.Tpm.DebugTracing.KSP] [FunctionEnd] PartA_PrivTags=16777216, Name=ProviderSignHash, HResult=The requested salt size for signing with RSAPSS does not match what the TPM uses. (0x40290423)Compare this with a working machine:
[TPM version: PCP_PLATFORM_TYPE: "TPM-Version:2.0 -Level:0-Revision:1.38-VendorID:'NTC '-Firmware:458754.131072"]11/15/22-09:49:31.3205956 [Microsoft.Tpm.DebugTracing.KSP] [FunctionEnd] PartA_PrivTags=16777216, Name=DetectPSSPaddingSalt, HResult=The operation completed successfully. (0x00000000)
11/15/22-09:49:31.3206060 [Microsoft.Tpm.DebugTracing.KSP] [FunctionEnd] PartA_PrivTags=16777216, Name=TpmKey20Rsa::SignHash, HResult=The operation completed successfully. (0x00000000)
11/15/22-09:49:31.3206072 [Microsoft.Tpm.DebugTracing.KSP] [FunctionEnd] PartA_PrivTags=16777216, Name=ProviderSignHash, HResult=The operation completed successfully. (0x00000000)The possible solutions are:
- Upgrade the TPM to a higher revision (if available)
- Disable the RSA-PSS signature algorithms on the client
- Use a certificate that does not use RSA signature algorithms
We've been attempting to source a TPM upgrade to get us to subversion 1.38 on affected devices.