Always On VPN IKEv2 - Poor Performance over WiFI?

Piotr Mikqus 6

We have a similar problem in our organization as some have written here before.
Unfortunately there is still no solution.
During a VPN connection, the bandwidth when working in a WIFI connection is very low.

Wired file copies are in the 10 MB/s range.
Wireless file copies are in the 1 MB/s range.

This applies to all home users.
This is not an individual situation.
Has anyone already encountered this and perhaps found a solution?

Sunny Qi 10,896 Reputation points Microsoft Vendor

2021-04-19T04:05:28.753+00:00

Please try to accept the replies which help you. It will encourage the person who help you. Appreciate your understanding. :)

8 answers

Robert Castles 31 Reputation points

2021-10-29T19:40:37.377+00:00

@Thomas Gusset @Gary Nebbett SSTP works. No notable networking issues. I did have to disable revocation check on the client as we are self signed here.
I'll look into what it takes to have a "Revocation Service".
Please sign in to rate this answer.
Thomas Gusset 36 Reputation points

2021-10-29T21:15:41.453+00:00

you can use Let's Encrypt certificate, see https://directaccess.richardhicks.com/2021/10/04/always-on-vpn-sstp-with-lets-encrypt-certificates/
Sign in to comment
Thomas Gusset 36 Reputation points

2021-10-29T18:19:50.153+00:00

In my frist post I wrote what transfer rates we see. What transfer rates do you observe?
AoVPN can also be used with SSTP instead of IKEv2. With SSTP we didn't see the bad performance.
Please sign in to rate this answer.

1 person found this answer helpful.
Robert Castles 31 Reputation points

2021-10-29T18:25:48.933+00:00

With the IKEv2 we are seeing caps around 350Kbps, vs a normal experience of 15Mbps+

That is very interesting about the SSTP option, I had not even considered it as the guides shy away from it for some reason.

I will add trying SSTP to the list of things to try. Thanks!

Thomas Gusset 36 Reputation points

2021-10-29T21:19:17.263+00:00

IKEv2 uses ESP over UDP. You might check if your edge firewall supress havy UDP traffic (UDP flood detection).This might would result in poor VPN performance.
Sign in to comment
Robert Castles 31 Reputation points

2021-10-29T17:54:34.997+00:00

By AoVPN, I'm referring to the IKEv2 and certificate auth based always on VPN whose setup is described here https://www.youtube.com/watch?v=aZ-thDAfuBM, referenced here https://learn.microsoft.com/en-us/windows-server/remote/remote-access/vpn/always-on-vpn/deploy/always-on-vpn-deploy, without DirectAccess.

By "MSVPN" I'm referring to the "on demand" PPTP VPN services from Microsoft, generally available since around 2000. https://en.wikipedia.org/wiki/Point-to-Point_Tunneling_Protocol
Please sign in to rate this answer.

0 comments No comments
Sign in to comment
Gary Nebbett 5,721 Reputation points

2021-07-14T15:06:39.953+00:00
Hello All,

Together with Thomas. I investigated this problem and we believe that we have found the explanation for the poor performance. If the explanation is correct, which we believe it probably (and mostly) is, then there are no practical workarounds.

I placed a more detailed analysis of our work at: http://gary-nebbett.blogspot.com/2021/07/slow-performance-of-ikev2-built-in.html.

In summary, there are two weaknesses in Microsoft components:

The implementation of the TCP CUBIC congestion control mechanism.

The distribution of work to worker threads by AgileVpn.sys (the IKEv2 WAN Miniport driver).

It takes a "third" ingredient to trigger the problem: the design of the network adapter device driver - in particular when it first indicates the arrival of a packet to NDIS. If the device driver directly indicates the arrival from its interrupt triggered DPC then everything works well. However, if the device driver defers the indication to a system worker thread then performance of the IKEv2 VPN declines sharply.

The sometimes observed and noted wired/wireless relationship between fast/slow IKEv2 VPNs is just a coincidence - it is the driver design that makes the difference. On my laptop where the built-in wired adapter is fast and the built-in wireless adapter is slow, a USB wireless network adapter is also fast.

Gary
Please sign in to rate this answer.

1 person found this answer helpful.
ZigZagMan 1 Reputation point

2021-09-28T18:44:58.177+00:00

Thank for the very detailed GaryNebbett. I have had 'some' success in both improving performance (and stability) by updating some Intel NIC drivers. I still am facing stability issues in keeping tunnels up but the driver appears to assist with throughput on some adapters.

Robert Castles 31 Reputation points

2021-10-29T15:14:36.9+00:00

Gary, thank you so much for your analysis. It is very detailed and helpful.

@Microsoft, we are attempting to migrate from MSVPN server to MS Always On VPN services, but we are unfortunately unable to proceed. Our MSVPN server is roughly 15-20 times faster than the AOVPN service on the same infrastructure. There no errors, logs or otherwise that we can find that indicate any particular issues.

It would be helpful to know if this issue is known and has any resources assigned to resolve.

Gary Nebbett 5,721 Reputation points

2021-10-29T15:56:55.747+00:00

Hello @Robert Castles ,

Were/are you using IKEv2 in your old/existing set-up? If IKEv2 was used then the problem that I described is probably not the cause - it is independent of the server side.

If your old set-up was not using IKEv2 (perhaps it used L2TP/IPsec) then we could explore if there is any easy way to confirm whether you are encountering the packet reordering problem without sharing trace data (confidential information).

Gary

Robert Castles 31 Reputation points

2021-10-29T16:19:32.36+00:00

EDIT: I'm not certain if our existing is using IKEv2. On the client side we are configured like this, below. On connection, it reports as WAN miniport (PPTP)

Thomas Gusset 36 Reputation points

2021-10-29T16:30:49.397+00:00

What do you mean with 'MSVPN Server' and 'MS Always On VPN services'?
AoVPN uses RRAS on the server side.

Gary Nebbett 5,721 Reputation points

2021-10-29T17:39:45.44+00:00

Hello @Robert Castles ,

On a system that still has the old configuration, you could try establishing a VPN connection and then start ncpa.cpl (the Network Connections Control Panel Applet) - that should show which VPN protocol is in use:

From what you have written, it is possible/probable that you have previously used PPTP and that, with the switch to AOVPN, IKEv2 and SSTP are now used instead. If so, that would mean that you could potentially be encountering the problem that I described.

Since you don't know for sure which VPN protocol has been used until now nor how to determine that information, it is difficult to think of a method of definitely identifying the cause of the problem without sharing trace data.

If you can access SMB file shares via the VPN, then one thing that you could try is copying a large file both to and from a file share. The problem that I described is asymmetric (only affects upload performance), so if the upload speed is much less than the download speed then this would be a hint.

Gary

Robert Castles 31 Reputation points

2021-10-29T17:48:11.347+00:00

Ok, got better details. Working VPN is WAN Miniport PPTP.

There are significant limitations with the IKEv2 much like you stated, except copying files to or from a Windows File Server time out and are generally capped right at 350Kbps, very close in speed to something someone else has noted. Downloads are more consistent, sticking at 350K, uploads are much worse, going from 0kbps for extended periods to 1.2Mbps, vs the PPTP VPN which easily gets 15Mbps with no interruptions.

Other services are affected, like RD, which is sluggish and laggy.

Gary Nebbett 5,721 Reputation points

2021-10-29T18:19:13.157+00:00

Hello @Robert Castles ,

The decrease in download performance is not a predicted (nor observed (by me, at least)) consequence of the "packet reordering" problem.

Some Event Tracing for Windows (ETW) tracing would be the next step in understanding what is happening, but that would reveal at least VPN endpoint information and some information about internal network addressing and naming.

I am willing and interested to understand the cause of your performance problems, but that is probably not compatible with your security requirements. Let me know if you have any flexibility in this respect.

Gary
Sign in to comment
Thomas Gusset 36 Reputation points

2021-07-06T13:02:09.15+00:00

Hi
we have exactly the same issue.
Good SMB2 performance (around 10 MByte/s) if client is connected via LAN.
Poor SMB2 performance (around 1 MByte/s) if client is connected via WLAN (2.3 or 5 GHz).
We don't see the issue if we use SSTP instead of IKEv2.
Measuring Internet speed shows no significant difference between LAN and WLAN (force tunnel, speedtest.net).
Curiously we have an other customer where we don't see this issue (good SMB performance with LAN and WLAN).
Configuration is exactly the same.
Internet bandwidth is also not a bottleneck (> 200/200 MB/s)

Any ideas?

Thanks, Thomas
Please sign in to rate this answer.

1 person found this answer helpful.
Gary Nebbett 5,721 Reputation points

2021-07-06T13:25:55.933+00:00

Hello @Thomas Gusset ,

The suggestion that I made to Piotr still stands - if you want to investigate the behaviour by means of network tracing then I am happy to help.

Gary

Thomas Gusset 36 Reputation points

2021-07-06T15:18:21.28+00:00

Hi Gary
thanks for that offer. I sent you a link request on LinkedIn.

Thomas

Gary Nebbett 5,721 Reputation points

2021-07-06T18:14:32.47+00:00

Hello Thomas,

Thanks for that - I have accepted.

I live in Bettingen, Kanton Basel-Stadt, so our active times and waking hours should (largely) coincide (I am retired). I cycled within just a few tens of metres from your premises last August (Route 35 - we cycled from Feldkirch to Uznach that day) :-)

I need to first perform a quick test on my own equipment to make sure that my first trace proposal makes sense (i.e. delivers useful information) - if your client systems have a relatively new version of Windows, then a combination of the Microsoft-Windows-PktMon and Microsoft-Windows-TCPIP ETW providers will probably be the starting point.

Your line of business is a great help too - highly technical with a deep understanding of security issues - I am looking forward to this collaboration.

Gary
Sign in to comment