L2TP over IPSec VPN doesn't work on Windows 11

Filip Zybała 5 Reputation points
2024-02-25T09:20:04.9233333+00:00

Hi!
I have a VPN server which is configured to handle L2TP over IPSec VPN. There are bunch of computers which use this VPN and everything works for them. I can just create the VPN connection using the UI. Today I got another Dell notebook with Windows 11 and ...guess what... VPN doesn't work. The error which I get is 809.
I started to dig into this topic, tried setting the registry keys, modifying all parameters of the connection, updating drivers, installing all Windows Update updates, reinstalling miniports and couldn't fix it. I decided to run Wireshark to see at which point the VPN is broken. I saw interesting output: There are no packets related to IPSec like ISAKMP. The VPN goes directly to L2TP part and fails obviously as 1701 port is blocked on firewall. As I said, I already tried to set the ProhibitIpsec registry key. I tested the connection with Wireshark on other Win11 machine and the connection starts correctly from ISAKMP packets. I feel like I read entire internet about this topic and couldn't fix it. Do you have any clues?

Windows 11
Windows 11
A Microsoft operating system designed for productivity, creativity, and ease of use.
10,675 questions
0 comments No comments
{count} votes

4 answers

Sort by: Most helpful
  1. Filip Zybała 5 Reputation points
    2024-02-25T22:47:50.1766667+00:00

    Update:

    I've got some tracings from the connection and in RasIpsec file there was such line:

    [4376] 02-25 22:04:55:158: Failed to generate certificate list. rc=0x103, Count=0, MyStoreEmpty=1

    1 person found this answer helpful.
    0 comments No comments

  2. Filip Zybała 5 Reputation points
    2024-02-25T22:59:01.8333333+00:00

    Ok, just got it. The whole L2TP/IPSec flow is broken when you have no certificates in Cert:/LocalMachine/My. It would be so nice if you could get some hints from VPN client instead of just: can't connect

    0 comments No comments

  3. Hania Lian 21,191 Reputation points Microsoft Vendor
    2024-02-27T07:01:40.7933333+00:00

    Hello, Thank you for posting in Q&A forum. Based on the business scenario, I believe this Microsoft Official Documentation will help on it: https://learn.microsoft.com/en-us/troubleshoot/windows-server/networking/troubleshoot-always-on-vpn#error-809-cant-establish-a-connection-between-local-machine-and-vpn-server Error 809 is more likely caused by the network device configuration issue. According to your statement, you have done a lot of troubleshooting steps on the Windows Client. Please kindly follow the traffic flow and verify if the traffic is dropped on any endpoint. Best Regards, Hania Lian

    ---If the Answer is helpful, please click "Accept Answer" and upvote it.

    0 comments No comments

  4. Deleted

    This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.


    Comments have been turned off. Learn more

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.