Site-2-Site VPN with whitelisted IPs
Dear azure team,
I setup S2S VPN from azure to an on-prem infrastructure. The status on azure portal says connected
. The tunnels are up on both sides but I am unable to pass traffic through it. Pinging
the private IP of the onprem systems is failing. nslookup
is failing too.
I have a hub-spoke infrastructure with firewall setup on hub-vnet virtual network and other virtual networks are peered with hub-vnet. I setup diagnostic settings to allow me checkout traffic flow within the tunnels. How are there spsecific ways to know what is blocking traffics from azure to on-premisses infrastructure. For context, this traffic is not even hitting the on-premise side at all.
By the way, the connection is allow us send traffic from our AKS through the tunnel to the On-premise infrastructure. The AKS itself is deployed in multiple subnets with virtual network
Azure VPN Gateway
Azure Firewall
-
KapilAnanth-MSFT 39,556 Reputation points • Microsoft Employee
2024-07-03T04:24:17.9533333+00:00 Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well.
From your verbatim,
- You have a Hub VNET with VPN Gateway connected to OnPrem
- There is a Firewall deployed in the Hub VNET
- Traffic from neither the Hub VNET nor the Spokes VNET is able to connect to the OnPrem servers.
- However, there is a AKS service that was able to pass traffic to OnPrem.
As next steps,
- Can you confirm if the AKS is deployed in the HubVNET or one of the Spoke VNETs?
- I see you have a Firewall in HubVNET
- Did you configure UDRs in the subnets with OnPrem ----> FirewallPrivateIP
- If so, can you check the Firewall logs to see if the traffic to OnPrem actually hits the Firewall or Not?
- Create a new VM in the HubVNET in a new subnet (without any RouteTable)
- Let's call it testVM
- From this testVM, please try to access your OnPrem resources and let us know if that succeeds.
Cheers,
Kapil
-
Seun Ore 40 Reputation points
2024-07-03T08:55:57.42+00:00 Thank you for response!
Can you confirm if the AKS is deployed in the HubVNET or one of the Spoke VNETs? Spoke VNETs
Did you configure UDRs in the subnets with OnPrem ----> FirewallPrivateIP.
This was done on onprem side. I have little visibility into the onprem side. But the tunnels were not up until we aligned on the UDR on both sides. However, on my side UDR that I setup on spoke vnet points to VPNGateway not the FirewallPrivateIP. (wrong?)Also VM setup in the hubvnet is unable to reach onprem server via pinging/nslookup
-
KapilAnanth-MSFT 39,556 Reputation points • Microsoft Employee
2024-07-04T14:28:56.5+00:00 For a Spoke Virtual Network,
- There is no need to use a Route Table in case you want to route traffic "directly" to the OnPrem Azure VPN Gateway
- All you have to do is enable Gateway transit.
- See : Configure VPN gateway transit for virtual network peering
- Can you confirm if gateway transit is enabled in the SpokeVNET?
- Without any RouteTables, if the VMinHub is unable to ping/reach the OnPrem servers indicate there could be firewall in OnPrem blocking this traffic.
- Can you confirm there was no route table in the subnet of the VMinHub
- If yes, from the VMinHub,
- Check NIC Effective Routes and let me know what is the nextHop for the OnPrem range?
- Check IP flow verify
- Virtual Machine : VMinHub
- NIC : <DEFAULT>
- Protocol : TCP
- Direction : OutBound
- Local IP : <DEFAULT>
- Local Port : 1234
- Remote IP : <ONPremServerIP>
- Remote Port : <ONPremServerPort>
- And share the results.
- You can get the same results using NSG diagnostics as well
P.S : If you want traffic to go via Azure Firewall before reaching the VPN Gateway and then OnPrem, the nextHop should be Firewall IP
-
Seun Ore 40 Reputation points
2024-07-05T13:48:52.2333333+00:00 Thank you @KapilAnanth-MSFT for response!
Can you confirm if gateway transit is enabled in the SpokeVNET? **Yes this was enabled.
**
Actually the biggest challenge is that I am unable to send traffic into my own tunnel.
The application is running on azure AKS with the ingress loadbalancer whitelisted on the onprem side. For me in particular, I struggled to get traffic inside the tunnel. I initiate traffic by going into the kubernetes pod and initiate ping traffic to the private IP on the onprem side.
My efforts so far:
- creates a route to Onprem CIDR range as Address prefix and VirtualNetworkGateway as next hop. Then, I associated this to subnet where AKS is deployed. This obviously did not work as you have mentioned in your post
- I enabled Gateway transit on the SpokeVnet
The main is that I don't know why traffic is not reaching the tunnel on my side.
-
Seun Ore 40 Reputation points
2024-07-05T14:01:56.25+00:00 By the way, I have limited visibility into Onprem side of this but I could see the mappings between our IP CIDR range and their own and this is correct. When I run tracert command from a VM in hub-vnet on azure to the Onprem IP, it failed. Again this is understandable because in the first place, I can't get any data into my own tunnel.
tracert 197.210.3.xxx
Tracing route to 197.210.3.xxx over a maximum of 30 hops
1 * * * Request timed out.
2 * * * Request timed out.
3 * * * Request timed out.
4 *
-
KapilAnanth-MSFT 39,556 Reputation points • Microsoft Employee
2024-07-05T16:09:22.58+00:00 When you mentioned, "By the way, the connection is allow us send traffic from our AKS through the tunnel to the On-premise infrastructure."
- I was under the impression that AKS traffic was passing via the tunnel to the OnPrem
- Is it not the case? Did you simply meant that the OnPrem firewall is configured to "allow" the traffic?
Next steps:
- Can you also please share the NIC Effective Routes and IP flow verify / NSG diagnostics as requested?
- In addition to enabling Gateway transit on the SpokeVnet, can you also confirm if you enabled "Allow gateway or route server in 'Hub-vnet' to forward traffic to the peered virtual network" in the HubVNET
Cheers,
Kapil
-
Seun Ore 40 Reputation points
2024-07-07T17:28:34.04+00:00 Thank you @KapilAnanth-MSFT for these responses!
I was under the impression that AKS traffic was passing via the tunnel to the OnPrem:
This is exactly what I am set to achieve, but currently failing because there is no traffic in the tunnel even with connection statusconnected
.NIC effective routes:
IP Flow Verify
**NSG diagnostics
**
And yes I enabled gateway or route server in 'Hub-vnet' to forward traffic to the SpokeVnet" **during vnet peering setup. This has been the case even before now.
**_
The whole setup was pretty straight-ward until we got to a point whereData in
andData out
show zeros on the Overview page of the connection. That is, no traffic in the tunnel. I feel the responsibility to at leat get traffic into my tunnel before raising some dust with the Onprem Guys if responses are not coming through theData in.
___
-
Seun Ore 40 Reputation points
2024-07-07T20:51:26.8833333+00:00 And by the way, I run network watcher for VPN troubleshooting for this particular connection and got the following result:
Connectivity State : Connected
Remote Tunnel Endpoint : 41.220.79.x
Ingress Bytes (since last connected) : 0 B
Egress Bytes (since last connected) : 0 B
Ingress Packets (since last connected) : 0 Packets
Egress Packets (since last connected) : 0 Packets
Ingress Packets Dropped due to Traffic Selector Mismatch (since last connected) : 0 Packets
Egress Packets Dropped due to Traffic Selector Mismatch (since last connected) : 60 Packets
Bandwidth : 0 b/s
Peak Bandwidth : 0 b/s
Connected Since : 7/7/2024 8:40:55 PM
PeakPackets : 0
TotalFlowCount : 0
Throttle : False
DeviceId :
I am absolutely sure that the IP mappings match on both azure and onprem.
And also, should the address space of my kubernetes cluster's subnet be replaced with the Public Ingress IP instead?
-
KapilAnanth-MSFT 39,556 Reputation points • Microsoft Employee
2024-07-08T10:35:03.98+00:00 Thanks for the info.
Looking at NIC Effective Routes and IP flow verify / NSG diagnostics, Azure side configurations look good.
For more details, you can consider using Azure VPN Gateway diagnostic logs.
To troubleshoot further, we will need a specialized 1:1 session, where a support engineer can have a screen share session to pinpoint the issue. If you have a support plan you may file a support ticket, else please do let us know, we will try and help you get a one-time free technical support.
Cheers,
Kapil
-
Seun Ore 40 Reputation points
2024-07-08T11:31:55.18+00:00 @KapilAnanth-MSFT thank you very much!
Yes, I have looked keenly at VPN diagnostic logs and in fact firewall logs. I didn't see solutions here too.
We do not have support plan, unfortunately. We used to get some support before the end of last month, June. But that has changed now and I have no idea why. We are on Enterprise Agreement billing.
-
KapilAnanth-MSFT 39,556 Reputation points • Microsoft Employee
2024-07-10T05:48:56.1366667+00:00 In that case, please share the subscription ID of the VPN Gateway in private messages.
I shall check if we can enable one-time free technical support.
Cheers,
Kapil
Sign in to comment