RDP Connection issue in Windows11

Question

Dear Team,

Windows 11 pc is trying to connect to a Server 2019 via RDP, the primary error was " an internal error occurred". After changing the registry and local security settings, a new error is showing "An authentication error has occurred. The specified data could not be decrypted. Again changed Oracle encryption in the registry but the result was the same. Kindly help if someone knows a solution.

Thanks,

Shabin

Answer 1

Hello Shabin Suresh1，

Thank you for posting in Microsoft Community forum.

Based on the error message you provided, please try the following steps to make modifications:

1. Check patch levels

Make sure both the Windows 11 PC and Server 2019 have been fully updated with the latest patches and updates.

2. Check corresponding group policy configuration and registry key values.

a. Group policy configuration method: Please navigate to the following policy path: Computer Configuration -> Administrative Templates -> System -> Credentials Delegation;

Find the following setting: "Encryption Oracle Remediation", and configure it as follows:

Enable Encryption Oracle Remediation, and select "Vulnerable" as the protection level.

b. Registry modification method (please backup before modifying) To modify the CredSSP registry of the RDP client, a restart is required for the changes to take effect.

Please open cmd with administrator privileges and run the following command to set it up: reg add HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\CredSSP\Parameters /v AllowEncryptionOracle /t REG_DWORD /d 2 /f

3. If the above steps do not solve the issue, please follow the steps below:

Check the security settings in group policy;

navigate to Computer Configuration -> Windows Configuration -> Security Settings -> Local Policies -> Security Options, and check the following two policies:

Network security: Restrict NTLM: Add remote server exceptions for NTLM authentication

If this policy is set to "Deny all account", please change it to blank;

Network security: Restrict NTLM: Incoming NTLM traffic

Please set this to "Allow all".

I hope the information above is helpful.

If you have any question or concern, please feel free to let us know.

Best Regards,

Haijian Shan

Answer 2

Hi Haijian Shan,

Thank you for your support.

I did all the things still same issue.

Noticed another thing, when changing Local Security Policies > Security Options > System Cryptography: Use FIPS-compliant algorithms for encryption, hashing, and signing, If this is disabled "an internal error occured" from RDP and Enabled will show an authentication error, The data could not be decrypted.

Kindly advise.

Best Regards,

Shabin

Answer 3

Hello   Shabin Suresh1，

Thank you for your reply.

Based on the new situation you provide , you can try the following steps:

1. Ensure that the Remote Desktop Services client and server are both using the same encryption algorithms. You may need to adjust the settings on both the client and server to ensure that they are compatible.
2. Check the event logs on the server to see if there are any error messages related to the authentication issue.
3. Try enabling the "Use FIPS-compliant algorithms for encryption, hashing, and signing" policy again, but this time, ensure that all necessary updates and patches are installed on the system.

Best Regards,

Haijian Shan

Answer 4

Discovered this same issue in the Fall of 2023 and have determined it is isolated to Win11 22H2 RDS connections to Win2019 Server; however, it does not affect all 2019 servers in our enviroment which makes it even more difficult to troubleshoot. We have had an open case with Microsoft on this for several months now and have collected more logs and traces than anyone could ever imagine and still no fix. Microsoft engineer sent us a "work around" where he had us do the following"

"On our last meeting we were able to fix it by changing the .exe and .dll of mstsc so we confirmed that the never version may have a bug of configuration change within the code.

We copied c:\windows\system32\mstsc.exe and mstscax.dll from a Windows 11 21H2 machine to the same location on your 22H2 machine and it worked properly on a machine which is not working with the 22h2 msrtsc.exe and dll.

So this is the proof that this is the scenario in which WIN 11 22H2 Remote Desktop Connection tool might be buggy or corrupted. We still looking into this at the moment. We appreciate your patience and effort ton this case."

Copying the mstsc.exe and .dll files from previous version of Win11 was not a viable solution because it had issues of its own. Using mRemote works just fine and we also performed in-place upgrades to Windows Server 2022 and RDS sessions work once the OS is upgraded to 2022. There is something unique with Windows Server 2019 and Windows 11 22H2 and unfortunately even though the issue is well documented, there still isn't an official fix from Microsoft on this. It has forced my organization to delay our planned project to update all our workstations to Windows 11 unfortunately. I have come acrosss a few other strange issues with Windows 11 and at this point that may also be bugs as well, Windows 10 appears to be far more stable and operational than Windows 11.