Vpn Gateway drops ports

Question

Vpn Gateway drops ports

Marco Mancini 60

hope this message finds you well. We currently have a VPN connection established between our on-premises Palo Alto firewall and the Azure VPN Gateway. However, we've encountered an issue where certain traffic appears to be dropped by the VPN gateway.

Specifically, when attempting to reach our on-premises destination on port TCP 3660 from VMA in Azure, the traffic not only fails but also does not appear in the on-premises firewall logs, suggesting that it may be blocked at an earlier stage in the connection path. The flow of traffic is as follows: VMA -> Palo Alto in Azure -> VPN gateway -> On-premises Palo Alto (serving as the VPN local gateway and firewall) -> destination Virtual Machine (VMB).

We have successfully established connectivity between VMA and VMB as evidenced by successful pings and traceroute tests. Additionally, connections towards ports 8080 and 443 are reaching the on-premises firewall successfully. However, we are encountering issues with port 3660, is not even reaching on prem firewall (no logs, no packets)

Our network security group (NSG) and firewall rules have been configured to allow the relevant traffic, and we have attempted to resolve the issue by resetting the connection and the VPN Gateway, unfortunately without success. It's important to note that no traffic selectors are set up on both sides.

Any assistance or guidance you can provide to help resolve this issue would be highly appreciated.

Thank you for your attention and support.

Best regards,

Marco

KapilAnanth-MSFT 49,621 Reputation points Microsoft Employee Moderator

2023-10-19T05:40:04.0366667+00:00
@Marco Mancini

Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well.

I understand that you are experiencing packet drop/connectivity issue while accessing on-premises destination TCP port 3660 from a VM in Azure.

Looking at your verbatim, It looks like you are facing a firewall issue and not a packet drop issue.

The network path is:

VM A -> Palo Alto in Azure (NVA) -> VPN gateway -> On-premises Palo Alto -> VM B (OnPrem server)

The best way to troubleshoot this is

To collect packet captures at every hop above and check if the packet was received in the hop or not.

You can use NetMon or Wireshark in VM A and Configure packet capture for VPN gateways tutorial for VPN gateway

For your NVA and OnPrem VPN Device, you should be able to collect captures as specified by your vendor.

Before doing the above,

I see you have mentioned that logs are not visible on the OnPrem VPN Device/Firewall

But what about the NVA in Azure?

Is it logging the traffic for destination port 3660

Is it visible in NVA and is the NVA allowing the traffic?

Can you bypass the NVA and give it a try?

i.e., in the VM A subnet's route table, add a route such that
<IP Address of VM B> ----> VPN Gateway

To check NSG , you can either use IP flow verify or NSG diagnostics

This will "simulate" traffic and will point out if a NSG rule blocks/allows a packet

In case, this steps did not help, you must do a packet capture on all the individual hops.

Please share the results of the above before proceeding with a capture

Cheers,

Kapil
Marco Mancini 60 Reputation points

2023-10-19T06:06:25.1033333+00:00
Hi Kapil, many thanks for your answer.

But what about the NVA in Azure?

Is it logging the traffic for destination port 3660 - yes, it is
- Is it visible in NVA and is the NVA allowing the traffic? - **yes, it is**

Can you bypass the NVA and give it a try?

i.e., in the VM A subnet's route table, add a route such that
<IP Address of VM B> ----> VPN Gateway Done, same result. When doing some connectivity tests, some ports (e.g. 8080, 443) are reaching on prem firewall and they appears in the logs. Others (e.g. 80, 3660) are not. There is no other firewall in the middle between VPN gateway and on prem firewall/vpn endpoint.

To check NSG , you can either use IP flow verify or NSG diagnostics
- This will "simulate" traffic and will point out if a NSG rule blocks/allows a packet **The NSG are allowing the traffic, and the tests of ip flow and nsg diagnostics are correct.**

I will proceed with a packet capture then.. thanks for your advices!
KapilAnanth-MSFT 49,621 Reputation points Microsoft Employee Moderator

2023-10-20T06:04:15.94+00:00

@Marco Mancini

Sure, please let us know how it goes.

Cheers,

Kapil
Marco Mancini 60 Reputation points

2023-10-20T06:07:45.4333333+00:00

Kapil, I have to apologize. We did not properly bypass the NVA during the first test (basically I created an asymmetric routing). When the NVA is bypassed, the traffic is passing through without any issue.
I then open a case with the vendor of the NVA, pending resolution (traffic marked as allowed not actually passing through).
Thanks again for your precious inpunts
KapilAnanth-MSFT 49,621 Reputation points Microsoft Employee Moderator

2023-10-20T06:18:52.3466667+00:00

@Marco Mancini

Thanks for keeping up updated.

Sure, I hope you get to the bottom of the issue with the vendor and mitigate it.

I have summarized my comments and posted it as a new answer.

I would highly appreciate if you could Accept the answer and close this thread.

Original posters help the community find answers faster by identifying the correct answer.

Thanks,

Kapil

Accepted answer

0 additional answers

Your answer

KapilAnanth-MSFT 49,621 Reputation points Microsoft Employee Moderator

2023-10-19T05:40:04.0366667+00:00

@Marco Mancini

Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well.

I understand that you are experiencing packet drop/connectivity issue while accessing on-premises destination TCP port 3660 from a VM in Azure.

Looking at your verbatim, It looks like you are facing a firewall issue and not a packet drop issue.

The network path is:

VM A -> Palo Alto in Azure (NVA) -> VPN gateway -> On-premises Palo Alto -> VM B (OnPrem server)

The best way to troubleshoot this is

To collect packet captures at every hop above and check if the packet was received in the hop or not.

You can use NetMon or Wireshark in VM A and Configure packet capture for VPN gateways tutorial for VPN gateway

For your NVA and OnPrem VPN Device, you should be able to collect captures as specified by your vendor.

Before doing the above,

I see you have mentioned that logs are not visible on the OnPrem VPN Device/Firewall

But what about the NVA in Azure?

Is it logging the traffic for destination port 3660

Is it visible in NVA and is the NVA allowing the traffic?

Can you bypass the NVA and give it a try?

i.e., in the VM A subnet's route table, add a route such that
<IP Address of VM B> ----> VPN Gateway

To check NSG , you can either use IP flow verify or NSG diagnostics

This will "simulate" traffic and will point out if a NSG rule blocks/allows a packet

In case, this steps did not help, you must do a packet capture on all the individual hops.

Please share the results of the above before proceeding with a capture

Cheers,

Kapil
Marco Mancini 60 Reputation points

2023-10-19T06:06:25.1033333+00:00

Hi Kapil, many thanks for your answer.

But what about the NVA in Azure?

Is it logging the traffic for destination port 3660 - yes, it is
- Is it visible in NVA and is the NVA allowing the traffic? - **yes, it is**

Can you bypass the NVA and give it a try?

i.e., in the VM A subnet's route table, add a route such that
<IP Address of VM B> ----> VPN Gateway Done, same result. When doing some connectivity tests, some ports (e.g. 8080, 443) are reaching on prem firewall and they appears in the logs. Others (e.g. 80, 3660) are not. There is no other firewall in the middle between VPN gateway and on prem firewall/vpn endpoint.

To check NSG , you can either use IP flow verify or NSG diagnostics
- This will "simulate" traffic and will point out if a NSG rule blocks/allows a packet **The NSG are allowing the traffic, and the tests of ip flow and nsg diagnostics are correct.**

I will proceed with a packet capture then.. thanks for your advices!
KapilAnanth-MSFT 49,621 Reputation points Microsoft Employee Moderator

2023-10-20T06:04:15.94+00:00

@Marco Mancini

Sure, please let us know how it goes.

Cheers,

Kapil
Marco Mancini 60 Reputation points

2023-10-20T06:07:45.4333333+00:00

Kapil, I have to apologize. We did not properly bypass the NVA during the first test (basically I created an asymmetric routing). When the NVA is bypassed, the traffic is passing through without any issue.
I then open a case with the vendor of the NVA, pending resolution (traffic marked as allowed not actually passing through).
Thanks again for your precious inpunts
KapilAnanth-MSFT 49,621 Reputation points Microsoft Employee Moderator

2023-10-20T06:18:52.3466667+00:00

@Marco Mancini

Thanks for keeping up updated.

Sure, I hope you get to the bottom of the issue with the vendor and mitigate it.

I have summarized my comments and posted it as a new answer.

I would highly appreciate if you could Accept the answer and close this thread.

Original posters help the community find answers faster by identifying the correct answer.

Thanks,

Kapil

Answer 1

@Marco Mancini

Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well.

I understand that you are experiencing packet drop/connectivity issue while accessing on-premises destination TCP port 3660 from a VM in Azure.

Looking at your verbatim, It looks like you are facing a firewall issue and not a packet drop issue.

The network path is:

VM A -> Palo Alto in Azure (NVA) -> VPN gateway -> On-premises Palo Alto -> VM B (OnPrem server)

To troubleshoot this, I suggested,

To collect packet captures at every hop above and check if the packet was received in the hop or not.

Also, validate :

Is the NVA in Azure logging the traffic for destination port 3660.
- Is it visible in NVA and is the NVA allowing the traffic??
Can you bypass the NVA and give it a try?
To check NSG , you can either use IP flow verify or NSG diagnostics
- This will "simulate" traffic and will point out if a NSG rule blocks/allows a packet

Meanwhile, you informed us the issue is not reproducible without the NVA in picture, and you have contacted the NVA vendor.

Hope you isolate the issue and resolve it.

Thanks,

Kapil

Please Accept an answer if correct.

Original posters help the community find answers faster by identifying the correct answer.

Marco Mancini 60 Reputation points

2023-10-20T13:00:54.7766667+00:00

Hi Kapil,
after double check it came that the issue was on Azure side: the UDR associated with VPN Gateway was having the Propagate gateway routes set on NO. After changing it on YES, the issue was solved.
I wanted to add this comment, in case could be useful for someone else.
Thanks again
Marco
KapilAnanth-MSFT 49,621 Reputation points Microsoft Employee Moderator

2023-10-23T05:45:25.0266667+00:00

Thanks for the update @Marco Mancini .

Sure, it would help the community

Share via

Vpn Gateway drops ports

0 additional answers

Your answer