[MAC-RRAS(VPN)] - "Negotiation Timed Out" for Always-On VPN (IKEv2)

tn-57-gs 26

Error: "CoId={************}: The following error occurred in the Point to Point Protocol module on port: VPN2-127, UserName: <Unauthenticated User>. Negotiation timed out

VPN Server: MS 2019 Server RRAS
NPS Server: MS 2019 Server NPS
Windows Clients: works flawlessly EAP-PEAP with Smartcard certificate (user cert)
Mac Client: Fails.

I have already raised a support case

2110040040003804

with MS and they denied to support macOS clients still I have not seen an article that states RRAS does not support macOS.

As you can see in the below screenshot, IKE_SA_INIT initiates a request from mac client and it even gets response back from the RRAS VPN server with SPI responder cookie and client sends IKE_AUTH request to the VPN server and server responds back but no further continuation in the flow, it breaks right there with the EvenID shared & screenshot above.

Apart from the VPN server event log and the packet trace I am not able to figure out what could be the reason behind. I have done all my research, changes to both client & server still no luck. please share your thoughts if you have experienced such issues.

2 answers

Gary Nebbett 5,721 Reputation points

2022-01-14T09:24:30.203+00:00

Hello @tn-57-gs ,

I wrote a blog entry on a related problem a few years ago: https://gary-nebbett.blogspot.com/2018/10/establishing-vpn-connection-from-macos.html

Since the "next" packet should come from macOS, the best place to start would be to examine the "racoon" log entries on the Mac.

From the Windows side, one could trace the IKE exchanges, using ETW (Event Tracing for Windows) and the "IKEEXT Trace Provider"; knowing the contents of the IKE_AUTH packet from Windows to Mac might give a hint about the problem. The "IKEEXT Trace Provider" is a WPP provider, so you probably won't be able to understand the trace data by yourself; if you are happy to share it then I would take a look.

Gary
Please sign in to rate this answer.
tn-57-gs 26 Reputation points

2022-01-15T20:29:02.76+00:00

Hi Gary

Thanks for your response.

I have attached the "IKEEXT Trace" from VPN server and also Wireshark logs from mac. please let me know how I can get the racoon logs and I tried with console app by filtering "Racoon" but there was no entries.

Hope attached logs helps.

Sujith

165340-ikeext-trc00.log 165278-msforumsupport.log

Gary Nebbett 5,721 Reputation points

2022-01-16T14:38:52.737+00:00

Hello Sujith,

The IKEEXT trace was probably started with a "keywords" value of 0; since it is a WPP provider, this means that no IKEEXT events were traced (it should be started with a keywords value of 0xFFFFFFFF and a level of 4). The events in the trace file are just equivalent of a Wireshark trace (Microsoft-Windows-NDIS-PacketCapture events).

I don't currently have access to a Mac; from my old notes, I used a commands like log show --debug --predicate 'process == "racoon"' and view /var/log/racoon.log to view the racoon events; I may have modified the racoon.conf file to increase the level of debug output.

The IKE_AUTH (MID=01) response in the traces is larger than that shown in your first image - its new size (1840 bytes) seems more reasonable to me.

It is just a guess, but you might find that racoon is logging the error: “Trust evaluate failure: [root AnchorTrusted BasicConstraints]” - the link in my first post discusses this potential problem.

Gary

tn-57-gs 26 Reputation points

2022-01-16T15:32:49.983+00:00

Thanks for sharing the command line to get the logs,

I can only see old logs and i tried to connect VPN and re-ran the command still it shows yesterday's one but not the one. I guess it may take some time.

Anyway, according to the old logs, PH1 initiated by the client but it fails with the below error, really not sure why.

*none message must be encrypted, status 0x1461, side 0 *
****racoon: [com.apple.networkextension:] CHKPH1THERE: no established ph1 handler found****

since attach option isn't working in this post, I have shared it using onedrive, please use the below link to download the racoon logs.

https://1drv.ms/u/s!AkVuXnM-kirQkPorAorQgJ4vTOCRyQ?e=JbycRD

please let me know if you need more details from me.

Gary Nebbett 5,721 Reputation points

2022-01-17T10:17:22.093+00:00

Hello Sujith,

I am not sure that the error message you mentioned is helpful; it always appears immediately (within a few microseconds) of the message "receive Information" - probably a reference to an IKEv2 INFORMATIONAL exchange; no messages of this type appear in the network traces near the connection attempts.

2022-01-13 22:55:00.783485+0100 0xba944 Default 0x0 16872 0 racoon: [com.apple.networkextension:] receive Information.
2022-01-13 22:55:00.783493+0100 0xba944 Error 0x0 16872 0 racoon: [com.apple.networkextension:] none message must be encrypted, status 0x1461, side 0

One can probably increase the level of racoon logging. The command man racoon.conf should give some hints about what can be done (perhaps add/edit log debug2).

Based on the step in the connection establishment when packet exchanges stop, the acceptability of the VPN server certificate to the macOS system is the most likely problem. Anything that can be done to verify the acceptability of the server certificate would be helpful (e.g. save a copy of that certificate on the macOS and use tools to verify the trust).

Gary

tn-57-gs 26 Reputation points

2022-01-18T12:57:43.883+00:00

According to apple, they are suggesting to refer "nessionmanager" for logs and i checked the logs,

NESMIKEv2VPNSession[Primary Tunnel:AZ_AOVPN_IKE:E1B0F261-1837-4F21-A81D-BB519E5B6386:(null)]: status changed to disconnected, last stop reason Authentication failed

It seems like fails due to authentication and not sure where to check the packet details from the clients end and I can see the incoming packets up to IKE_AUTH but it does not reach NPS, need to figure out why? what is missing in the packet?

either client side logs should give us a hint about it OR from the VPN's end and I know its not easy either.

Gary Nebbett 5,721 Reputation points

2022-01-19T10:24:49.113+00:00

Hello Sujith,

I think that you need to persevere with racoon logging first. Other layers possible report failures by lower layers in a misleading way.

Look at section 2.16 of RFC 5996:

Initiator Responder ------------------------------------------------------------------- HDR, SAi1, KEi, Ni --> <-- HDR, SAr1, KEr, Nr, [CERTREQ] HDR, SK {IDi, [CERTREQ,] [IDr,] SAi2, TSi, TSr} --> <-- HDR, SK {IDr, [CERT,] AUTH, EAP } HDR, SK {EAP} --> <-- HDR, SK {EAP (success)} HDR, SK {AUTH} --> <-- HDR, SK {AUTH, SAr2, TSi, TSr }

I am fairly sure that the last packet (which was sent by the server) was HDR, SK {IDr, [CERT,] AUTH, EAP }; the EAP attribute is just an EAP "start" message. The attribute most likely to have been found unacceptable by the macOS client is the server CERT.

Gary

tn-57-gs 26 Reputation points

2022-01-19T11:50:56.877+00:00

could you assist me setting up racoon logging further? please share a action plan so that I can perform on my mac client.

FYI, I even think the same, server is requesting for certificate from the client and the client does not OR unable to fulfill the requested action, yes, I should dig deeper to see why mac not responding to IKE_AUTH OR what is missing

Please assist me here with your action plan.

Sujith

tn-57-gs 26 Reputation points

2022-01-19T15:52:28.5+00:00

So you say server CERT is the problem, which one you are referring to SCEP User CERT OR Root CERT or VPN Cert?

FYI, we have our PKI and SCEP user certs are distributed via NDES & Trusted Root CERT are distributed using Intune. The same procedure is followed for windows devices and they are getting connected without a problem.

Gary Nebbett 5,721 Reputation points

2022-01-19T16:11:49.94+00:00

Hello Sujith,

As mentioned, I don't currently have access to a Mac; it is difficult to describe steps that I can't test and have not performed for a long time.

The certificate in the last message is the certificate for the VPN server; the checks performed by macOS will require the whole chain to the root certificate and the root certificate must be trusted by macOS.

Gary

tn-57-gs 26 Reputation points

2022-01-19T17:10:28.783+00:00

Hi Gary,

Thanks for your replies so far.

All the relevant certificates are set to "Always Trust' for EAP & IPSEC therefore I don't think there will be a problem with the trust factor but your statement makes me think about VPN server certificate, the VPN connection sting name is same as VPN server and the CERT was issued by the internal PKI but not signed by external CA. do you think this could also be problem for the mac to trust such certs?

the same is used on windows devices and there is no problem with it.

Sujith

tn-57-gs 26 Reputation points

2022-01-19T18:39:32.57+00:00

Like mentioned in the following link http://glozer.net/soekris/macosx-ipsec.html , there is a script that creates an IPSec tunnel which prevents a potential sniffer from analyzing network traffic based on the final destination of the packet, do you think running this script might give us more details what's happening in the backend while initiating connection. moreover I have also attached the racoon.cong "https://1drv.ms/u/s!AkVuXnM-kirQkPpbWJ2GcB_zmTmulg?e=7KORj4" from /etc/racoon/racoon.conf file fyi.

please suggest here if we can take this direction to check more insight logs from mac.

tn-57-gs 26 Reputation points

2022-01-19T19:03:17.32+00:00

There is also one another thing that runs in my mind, The IPsec policy must match on both the server and the client for an IKEv2 VPN connection to be successful, on windows we have PowerShell commands to set the custom IPSEC policies but not sure if it is required to be set on macOS as well.

my VPN server's IPSEC policy is as follows:

AuthenticationTransformConstants SHA256128

CipherTransformConstants AES128

DHGroup Group14

EncryptionMethod AES128

IntegrityCheckMethod SHA256 -

PFSgroup PFS2048

SALifeTimeSeconds 28800

MMSALifeTimeSeconds 86400

SADataSizeForRenegotiationKilobytes 1024000

Is it really needed to set the above ipsec policy on mac client as well? if so where this should be done, is it in the raccon.conf or it should be the part .mobileconfig file.

Regards
Sujith

Gary Nebbett 5,721 Reputation points

2022-01-19T19:04:37.063+00:00

Hello Sujith,

As I have already, mentioned, setting a higher logging level on racoon might help.

The .conf file that you posted includes the following lines:

# "log" specifies logging level. It is followed by either "notify", "debug" # or "debug2". #log debug;

The "#" symbol means comment. If you were to add "log debug2" to the .conf file, then there should be more useful information in the logs; since there is currently no active "log" directive, we are probably just seeing the default (low) level of logging.

Gary

tn-57-gs 26 Reputation points

2022-01-19T23:18:02.567+00:00

Hi Gary

As you mentioned, I added the log level to "log debug2" and tried the below command along with --last switch to get the latest logs but unfortunately it returns result only when I specify --last 6h (which is 6 hours)

log show --debug --predicate 'process == "racoon"' --last 6h // gives results but older than 6 hours log

tried below

log show --debug --predicate 'process == "racoon"' --last 5m // no results returned when specified 5 minutes

Alongside in search for syntax, I came across this link https://www.daemon-systems.org/man/racoon.conf.5.html where they talk about how to configure the racoon config file. Is it really mandatory to modify authentication_algorithm, encryption_algorithm, remotehostname, radius, PH Group etc? I guess, during the initial packet exchange by default mac adapts to payloads exist in VPN server so there is no need to match between client and server?

correct me if I am wrong with my assumption mentioned above.

Regards
Sujith

Gary Nebbett 5,721 Reputation points

2022-01-20T09:57:40.03+00:00

Hello Sujith,

It might be necessary to send a signal to racoon (or to restart racoon) in order for changes in racoon.conf to take effect.

Once that has been done, you could try log show --debug without the 'process == "racoon"' predicate but selected by time (e.g. --last or --start or whatever) and make that available.

Mutually acceptable algorithms and parameters have to be negotiated for quick mode and main mode. The traces show that quick mode negotiation was successful (the encrypted packets in the trace are using the negotiated algorithms/parameters). Main mode negotiation has not yet completed; until the Mac is able/willing to send the next packet, we won't know for sure whether any configuration adjustments in this regard are necessary.

Gary

tn-57-gs 26 Reputation points

2022-01-20T10:26:10.45+00:00

HI Gary

Thanks for the clarity thrown to the question about initial handshake I asked in my previous question.

Yesterday itself, I tried restarting racoon using the below command still it does not show up the latest log entries.

sudo launchctl stop com.apple.racoon sudo launchctl start com.apple.racoon

still finding a way to get the latest logs.

Regards
Sujith

tn-57-gs 26 Reputation points

2022-01-20T19:43:08.767+00:00

After a long hunt on getting detailed information about IKE logs, I identified that my mac does not trust the certificate issued by an Internal CA and the reason is unknown, due to which mac failed to use the certificate for authentication. please see the screenshot below for your reference.

Now, i need to figure out why these certificates are not trusted even after setting it to TRUST ALWAYS for ALL USERS from the keychain app.

iRegards
Sujith

Gary Nebbett 5,721 Reputation points

2022-01-20T20:31:16.013+00:00

Hello Sujith,

I think that one should focus on establishing the root certificate authority as a trusted root authority. I don't think that any special steps are needed for the specific certificate of the VPN server,

Gary

Gary Nebbett 5,721 Reputation points

2022-01-21T09:48:14.763+00:00

Hello Sujith,

A potential problem might be that the VPN server certificate contains a CRL Distribution Points extension that only lists "unreachable" (for the Mac) distribution points (e.g. that use the ldap:/// scheme).

Gary

tn-57-gs 26 Reputation points

2022-01-22T12:18:26.22+00:00

In the VPN configuration profile set on my mac has an attribute "EnableCertificateRevocationCheck = 0" which means the mac will not perform CRL check during the connection attempt.

Ref: https://developer.apple.com/documentation/devicemanagement/vpn/ikev2

I suspect the problem seems com.apple.vpn.managed native app is somehow not able to get the needed cert from the key store OR certificate evaluation fails not sure why.

Regards
Sujith
Sign in to comment
Limitless Technology 39,351 Reputation points

2022-01-14T14:00:43.187+00:00
Hi @tn-57-gs

Some quick points to check out.

Check VPN server certificate has "server authentication" EKU

Check certificates are valid on the client, VPN server, and NPS server

Check the client, VPN server, and NPS server all have trusted root certificate from the DC (CA administrator)

Check the VPN server name on the client matches the VPN server certificate's subject name

Check appropriate port (1812, for RADIUS authentication) is open on VPN server and NPS server

Check the NPS server is reachable (ping-able) from the VPN server.

Here is a thread as well which discusses the same issue and you can try out some troubleshooting steps from this and see if that helps you to sort the Issue.

https://social.technet.microsoft.com/Forums/ie/en-US/0c81d9d6-19ff-407f-9206-26a17ecec532/quotnegotiation-timed-outquot-for-alwayson-vpn-ikev2?forum=ws2016

Please try the following articles to see if they could be of help.

https://blogs.technet.microsoft.com/rrasblog/2009/08/12/troubleshooting-common-vpn-related-errors/

https://social.technet.microsoft.com/Forums/ie/en-US/771bf5ec-7017-4fd3-9496-52137dfa616a/error-description-13801-ike-authentication-credentials-are-unacceptable?forum=winservergen

Hope this resolves your Query!!

------
--If the reply is helpful, please Upvote and Accept it as an answer--
Please sign in to rate this answer.
tn-57-gs 26 Reputation points

2022-01-19T11:42:24.663+00:00

Hi

Like I have mentioned already in this post, with the same configuration works very well on windows client and the issue is happening only on mac clients. clueless what could be the problem. besides, i have also cross-checked all of the checklist you have mentioned still the issue remains the same. already raised a MS support case and they also gave up with this issue.
Sign in to comment