The question at hand is… “Why is the TMG server sourcing packets with a secondary address instead of the primary address of the NIC?”

The answer to that question deals with the differentiation between the Network Stack in Server 2008 and above and Server 2003 and below. Server 2003\XP and below were based off the Weak Host Model. Basically, in a Weak Host Model the primary address of the adapter with a route that most closely matches the target IP address is used as the Source IP Address.

In server 2008\Vista and above we re-architectured the Network stack. It is based on the Strong Host Model. In the Strong Host Model the concept of a Primary IP Address doesn’t exist. To determine the IP address that is utilized it looks at the routing table to decide the proper NIC to utilize, then uses the process defined in RFC 3484 to choose the source IP for outbound packets. Here is a basic breakdown of how the windows implementation of RFC 3484 chooses an IP address:

Windows Source IP V4 address selection:

- Rule 1 Prefer same address (applies)

- Rule 2 Prefer appropriate scope (applies)

- Rule 3 Avoid deprecated addresses (applies)

- Rule 4 - Prefer home addresses - does not apply to IP v4

- Rule 5 Prefer outgoing Interfaces (applies)

- Rule 6 Prefer matching label - does not apply to IP v4

- Rule 7 Prefer public addresses - does not apply to IP v4

- Rule 8a: Use longest matching prefix with the next hop IP address. (not in RFC!)

"If CommonPrefixLen(SA, D) > CommonPrefixLen(SB, D), then prefer SA. Similarly, if CommonPrefixLen(SB, D) > CommonPrefixLen(SA, D), then prefer SB. "

*This says that the IP with the most high order bits that match the destination of the next hop will be used.

Note: Rule 8 - Use longest matching Prefix is similar to rule 8a except the match is with the destination IP address rather than the next hop IP address.

For example, use the following addresses as an example of choosing the longest matching prefix:

TMG Servers External IP Addresses:

192.168.1.14/24
&
192.168.1.68/24

Default Gateway:

192.168.1.127

Convert these addresses into binary:

192.168.1.14   = 11000000 10101000 00000001 00001110
192.168.1.68   = 11000000 10101000 00000001 01000100
192.168.1.127 = 11000000 10101000 00000001 01111111

The 192.168.1.68 address has more matching high order bits with the gateway address 192.168.1.127. This would cause the server to utilize what was originally defined as the “secondary” as the Source IP address of outgoing packets.

*For more information on RFC 3484 please refer to the following link: https://www.ietf.org/rfc/rfc3484.txt . Please note that IPv6 is referenced in RFC. Windows utilize the same process for IPv4 sourcing.  You can also review the following TechNet article for supported document details on the above information:

The functionality for source IP address selection in Windows Server 2008 and in Windows Vista differs from the corresponding functionality in earlier versions of Windows
https://support.microsoft.com/kb/969029

So now we know why your TMG server may be sourcing your packets with what you call your “Secondary IP Address”. It isn’t TMG at all. It is the default behavior of the server itself. Your server version is Server 2008 or above. The question is…

“Can I configure my Server 2008 or above in a way that it will only utilize the first IP address as a Source address?”

The answer to that is YES! There is actually a Netsh command that can be utilized to add IP addresses. In that command you use the “SkipAsSource” flag and it will no longer use the IP address you are adding as a Source IP Address. This means that you will have to temporarily remove the IP Address you are having the issues with then re-add them utilizing the Netsh command (This means you will have to have a maintenance window!). Here are examples of the command lines you will use:

Server 2008:
Netsh int ipv4 add address <Interface Name> <ip address> <subnet mask> skipassource=true

Server 2008 R2:

Netsh int ipv4 add address <Interface Name> <ip address> skipassource=true

* For details and prerequisites to utilize these commands please refer to the following articles:

All IP addresses are registered on the DNS servers when the IP addresses are assigned to one network adapter on a computer that is running Windows Server 2008 SP2 or Windows Vista SP2
https://support.microsoft.com/default.aspx?scid=kb;EN-US;975808

IP addresses are still registered on the DNS servers even if the IP addresses are not used for outgoing traffic on a computer that is running Windows 7 or Windows Server 2008 R2
https://support.microsoft.com/default.aspx?scid=kb;en-US;2386184

Keep in mind that I gave only one specific example where this may be causing an issue.  There may be other problems you run into where the Netsh entry I listed may help you out.  No telling… it may not even be on your TMG servers.  Maybe you see the issue on your UAG servers, or any other server for that fact. 

I hope the information provided helps out!

*Please do keep in mind as you read on that this information is only an example and will need to be adjusted and tested in every given scenario. There are many ways to accomplish this process. This blog was created to start you on your way in a thought process to complete your desired needs.

So what do I mean by “cover-up” pages? Well, think about the possibility of using the Form Fill process that UAG offers. This process is used when you are utilizing Forms Based Authentication on backend applications (such as SharePoint) and you don’t want your clients to have to provide credentials multiple times. You can have UAG fill in the credentials that were utilized to Authenticate on the Perimeter.

* For more on the Form Fill process please refer to the following articles:

https://blogs.technet.com/b/ben/archive/2010/09/02/uag-custom-form-login.aspx

https://blogs.technet.com/b/ben/archive/2010/01/23/custom-form-login-sso-how-to.aspx

All that said, still… what do I mean by “Cover-up” pages and why do we need them? Well, when you use the Custom Form Login process the client hits the back-end application server through UAG and gets posed with a Form Based Authentication page. UAG can then fill in that Form for the client a moment later giving a Single Sign-on (SSO) type of experience. The problem is that the client still sees the blank form and could possibly start trying to fill it in prior to UAG taking over and filling it in therefore causing an issue. Therein… the “Cover-up” page! It’s using the AppWrap process to tell UAG “When you see this specific application send a Form Based Authentication page to the client, replace it with a different page so the client doesn’t even see the Form.” The example I give below is nothing more than a white square that covers up the entire FBA page. Of course the Form Fill example I have given is only one manner in which the following AppWrap HTML information can be utilized.  Please know that for this example to work you already have successfully configured the Custom Form Login process. If you haven’t… then your clients will never see the Forms Based Authentication page and time out!

AppWrap explanation:

AppWrap (Application Wrappers) are customization files that enable UAG to search for specific types of HTTP Responses from back-end application servers and manipulate them. This UAG customization feature gives you a way to provide the external client with a different page than what was sent by the application server therefore giving them a better overall browsing experience.

*For more on configuring AppWraps please refer to the following Blogs:

https://blogs.technet.com/b/ben/archive/2011/05/12/appwrap-and-sra.aspx

https://blogs.technet.com/b/edgeaccessblog/archive/2009/11/17/appwrap-in-uag-what-s-new.aspx

In my example for a “Cover-up” page we need to have the AppWrap look for the Login page by both the application type, and the URL pattern. We then replace the HEAD of the page with added elements to create a white “Cover-up” square.

HTML Explanation:

We want to cover the entire screen, so we create an HTML element ("DIV", which stands or 'document divisions') that has no contents, and we set it to cover the screen. To do so, we create a style-sheet element called 'cover', and assign it to the blank DIV using the id attribute

<div id="cover"></div>

The style element itself is supposed to cover the entire screen with a white curtain, so we set it to have the following elements:

1) Color: white

2) Location: starts at the top-left corner of the screen

3) Size: cover 100% of the screen

This looks like this:

<style>

#cover {

                background-color: #ffffff;

                position: absolute;

                top: 0;

                left: 0;

                width: 100%;

                height: 100%;

                }

</Style>

The 'absolute' position attribute means that the position where it starts is not relative to where this DIV is located in the HTML, but absolute...starts at the very top and left position of 0.

In HTML, the line break has no meaning, so to make it easy for AppWrap, we simply remove those, and compress it all into one line that has both the STYLE tag and the DIV tag:

<style>#cover { background-color: #ffffff; position: absolute; top: 0; left: 0; width: 100%; height: 100%;}</Style><div id="cover"></div>

Final AppWrap (encoded):

* In the example below you will notice the Bold sections for APPLICATION_TYPE and URL. These two will be populated with specific information from your environment.

<APP_WRAP ver="3.0" id=" WhlFiltAppWrap_HTTPS.xml">

                <MANIPULATION>

                <MANIPULATION_PER_APPLICATION>

                                <APPLICATION_TYPE>Application_Type_Description</APPLICATION_TYPE>

                                <DATA_CHANGE>

                                                <URL>.*login\.asp.*</URL>

                                                <SAR>

                                                                <SEARCH encoding="base64">PEhFQUQ+</SEARCH>

                                                                <REPLACE encoding="base64">PEhFQUQ+PHN0eWxlPiNjb3ZlciB7IGJhY2tncm91bmQtY29sb3I6ICNmZmZmZmY7IHBvc2l0aW9uOiBhYnNvbHV0ZTsgdG9wOiAwOyBsZWZ0OiAwOyB3aWR0aDogMTAwJTsgaGVpZ2h0OiAxMDAlO308L1N0eWxlPjxkaXYgaWQ9ImNvdmVyIj48L2Rpdj4=</REPLACE>

                                                </SAR>

                                               <SAR>

                                                                <SEARCH encoding="base64">PGhlYWQ+</SEARCH>

                                                                <REPLACE encoding="base64">PEhFQUQ+PHN0eWxlPiNjb3ZlciB7IGJhY2tncm91bmQtY29sb3I6ICNmZmZmZmY7IHBvc2l0aW9uOiBhYnNvbHV0ZTsgdG9wOiAwOyBsZWZ0OiAwOyB3aWR0aDogMTAwJTsgaGVpZ2h0OiAxMDAlO308L1N0eWxlPjxkaXYgaWQ9ImNvdmVyIj48L2Rpdj4=</REPLACE>

                                                </SAR>

                                </DATA_CHANGE>

                </MANIPULATION_PER_APPLICATION>

</MANIPULATION>

</APP_WRAP>

Final AppWrap (plain text):

* This is just an explanation of the HTML encoded AppWrap given in plain text to try to make it a little more understandable. Please note: You MUST utilize the encoded version above in your custom AppWrap.

<APP_WRAP ver="3.0" id=" WhlFiltAppWrap_HTTPS.xml">

                <MANIPULATION>

                <MANIPULATION_PER_APPLICATION>

                                <APPLICATION_TYPE> Application_Type_Description </APPLICATION_TYPE>

                                <DATA_CHANGE>

                                                <URL>.*login\.asp.*</URL>

                                                <SAR>

                                                                <SEARCH><HEAD></SEARCH>

                                                                <REPLACE><HEAD><style>#cover { background-color: #ffffff; position: absolute; top: 0; left: 0; width: 100%; height: 100%;}</Style><div id="cover"></div></REPLACE>

                                                </SAR>

                                               <SAR>

                                                                <SEARCH><head></SEARCH>

                                                                <REPLACE><HEAD><style>#cover { background-color: #ffffff; position: absolute; top: 0; left: 0; width: 100%; height: 100%;}</Style><div id="cover"></div></REPLACE>

                                                </SAR>

                                </DATA_CHANGE>

                </MANIPULATION_PER_APPLICATION>

</MANIPULATION>

</APP_WRAP>

I hope the information I provided above helps out! Make sure to check out all the links I provided to the additional Blogs as they will get you a long way in your customization processes!

To find the version of UAG you are using, in your MMC, go to Help, and then "About …” and you will see the following information:

You can utilize the following data to determine what patch level you are currently at once you find your Version number:

* This article will be updated as new major patches are released.

** You can also review the following Blog to find out more information regarding the differences
between a “Private Fix”, a “Roll-up”, an “Update”, and a “Service Pack”:

https://blogs.technet.com/b/edgeaccessblog/archive/2010/12/06/types-of-updates-available-for-uag.aspx

Source:        Service Control Manager
Event ID:      7034
Level:         Error
Description:
The Microsoft Forefront TMG Firewall service terminated unexpectedly.  It has done this <times>.

And\Or

Source: Microsoft Forefront TMG Firewall
Event ID: 14057
Description:
The Firewall service stopped because an application filter module C:\Program Files\Microsoft Forefront Threat Management Gateway\w3filter.dll generated an exception code C0000005 in address <hex_address> when function CompleteAsyncIO was called. To resolve this error, remove recently installed application filters and restart the service.
 

It has been determined that applying TMG SP2 Rollup2 will resolve the issue.  To install Rollup 2 please refer to the following link:

Rollup 2 for Forefront Threat Management Gateway (TMG) 2010 Service Pack 2
https://support.microsoft.com/kb/2689195

 More information will be provided as it comes available.  Thank you for your patience in this matter.

* Please keep in mind that I am not suggesting that we recommend putting an Evaluation version of the TMG product into production environments. It is actually the exact opposite. It is meant for testing purposes only.

So all that said let’s assume that there’s a configuration of TMG that is an Evaluation version that you would like to keep up and running.  “What! Keep up and running! “   Yes… the Evaluation versions are time bombed.  You will see issues in running the firewall service once you have passed your Evaluation time period.  So, let’s talk about how you can minimize downtime and get these servers back up in an supportable RTM configuration.

What needs to be done?  Well, first you need to ask yourself:

“What type of configuration am I in?  Is it a Stand-alone server?  Is it a Stand-alone Array with 2 or more Firewall nodes?  Or is it an Enterprise Array with 2 or more Firewall nodes?”

Based on your answer just choose one of the following quick and easy steps:

Stand-alone Server:

This process is simple enough.  You have to reinstall the product, that isn’t a question.  You can’t just go out and purchase the product codes and put them in, so expect an install is going to be needed.  Here are the steps for a single server:

1. Export your server configuration (Make sure to include the confidential information as well as the user permission settings).

2. Uninstall the TMG product from the server by going into Programs and Features, highlighting Microsoft Forefront Threat Management Gateway, and choosing Uninstall.

3. Install Threat Management Gateway utilizing the RTM bits you have purchased.

4. Import your configuration you saved from step 1 above.

* Please keep in mind that you haven’t changed anything on the server itself. You haven’t removed any NICs, changed IP addresses, uninstalled certificates, etc. You have just uninstalled the TMG product and reinstalled it.

Stand-alone Array with 2 or more Firewall nodes:

Seeing that this is an actual Array configuration (even though not an enterprise array utilizing a separate EMS), you’ll notice a big difference in the steps below versus the steps above.  What’s nice is the steps I provide should keep you up and running as long as you are using NLB for load balancing purposes.  For this example I will be using 2 nodes in an array.

1. Start by exporting a full configuration of your Array for backup purposes in case any problems occur.

2. Determine which Node is your Array Manager.  This is important because you will need to begin all the work on the Firewall Node that is NOT the Array Manager.  To determine which of the two servers is your Array Manager just open your TMG MMC, highlight system on the left, then scroll over to the far right on your Servers tab located in the middle of your MMC.  You’ll see one server says “Array Manager” and the other says “Array Managed”.

3. Notate the version numbers that your servers are at.

* This is very important because you will need to make sure you install the proper updates to bring them back up to match. If you don’t have them at the same patch level you will see issues when you try to join back to the Array.

4. From the Managed Server, highlight the Array name on the left side of the MMC and choose “Disjoin Server From Array” on the far right.

5. Once the server is fully disjoined from the Array go ahead and uninstall the TMG product from the server by going into Programs and Features, highlighting Microsoft Forefront Threat Management Gateway, and choosing Uninstall.

6. Install Threat Management Gateway utilizing the RTM bits you have purchased.  Once the product has been reinstalled make sure to install all the needed updates to bring it to the same version as the current Array Manager.

7. Once the server is back up highlight “Forefront TMG (Server_ name) on left side of the MMC and choose “Join Array” from the far right.  Point to the Array Managers name.

8. Once you have fully joined back to the Array you will notice that all your rules and configurations have been pulled back over to your new RTM TMG server and everything should begin working again.

9. On the new RTM TMG server you just joined, highlight the Array on the left. In the Tasks tab on the far right choose “Set as Array Manager”.  This will make this new server take control of the Array Manager process from the Evaluation server that is still running.

10. Go over to the remaining Evaluation server and highlight the Array on the left side of the MMC and then choose “Disjoin Server From Array” on the right.

11. Once the server is fully disjoined from the Array go ahead and uninstall the TMG product from the server by going into Programs and Features, highlighting Microsoft Forefront Threat Management Gateway, and choosing Uninstall.

12. Install Threat Management Gateway utilizing the RTM bits you have purchased.  Once the product has been reinstalled make sure to install all the needed updates to bring it to the same version as the current Array Manager.

13. Once the server is back up highlight “Forefront TMG (Server_ name) on left side of the MMC and choose “Join Array” from the far right.  Point to the new Array Managers name.  This will cause the server to join the Array and pull down all the needed configurations. 

Enterprise Array with 2 or more Firewall nodes:

Believe it or not this is actually the easiest of the three different configurations.  The thing is… you don’t have to worry about updating your EMS (Enterprise Management Server) to RTM, just your firewall nodes.

1. Start by exporting a full configuration of your Array for backup purposes in case any problems occur.

2. Notate the version numbers that your servers are at.

* This is very important because you will need to make sure you install the proper updates to bring them back up to match. If you don’t have them at the same patch level you will see issues when you try to join back to the Array.

3. Choose the firewall node that you want to start your update to RTM on.  There is not a specific node that should go first.

4. From the server you chose in step 3… highlight the Array on the left side of the MMC and then choose “Disjoin Server From Array” from the far right.

5. Once the server is fully disjoined from the Array go ahead and uninstall the TMG product from the server by going into Programs and Features, highlighting Microsoft Forefront Threat Management Gateway, and choosing Uninstall.

6. Install Threat Management Gateway utilizing the RTM bits you have purchased.  Once the product has been reinstalled make sure to install all the needed updates to bring it to the same version as the current Array Manager.

7. Once the server is back up highlight “Forefront TMG (Server_ name) on left side of the MMC and choose “Join Array” from the far right.  You will be joining an EMS managed Array so you will need to point to the name of your existing EMS.  This will cause the server to join the Array and pull down all the needed configurations.

8. Once this server is back up and functioning as expected go to the second Firewall node that is still utilizing Evaluation bits and go through steps 4, 5, 6, and 7 just above.

So… at this point you should be up and running with RTM bits!  I hope the information I provided above helps out!  Keep in mind, ALWAYS GET BACKUPS PRIOR TO ANY WORK!  By backups I am referring to System State and TMG configuration backups (or exports).  If you run into any issues going from Eval to RTM know that there are support processes that can help.  In any case, you may still want to make sure this is all done in a maintenance window.  No need to cause an outage if something goes wrong!

Hello all!  It’s me, Brett Crane, from the Security teams.  I worked a really interesting issue with one of my customers recently and wanted to point out our findings and how to fix or work around the issue…

Problem:

We found that when we have a 2 node TMG Array (running Service Pack 1, Update 1 Rollup 4 and below) with NLB enabled on both the internal and external interfaces, Session Affinity doesn’t seem to be working properly. The peculiarity here is that the load balancing affinity issue is only seen when utilizing Web Publishing Rules. When server Publishing rules are utilized everything works fine.

When running NLB on both the internal and external interfaces of TMG we utilize what is termed as bidirectional affinity. Basically, bidirectional affinity enables NLB-enabled firewall arrays to support complex protocols requiring secondary connections and ensures that secondary connections are routed through the same firewall through which the initial client request was made. This prevents connections from being routed to an array member that has no knowledge of the initial connection. This is the piece that was being seen as a suspect from the start.

Data Analysis:

TMG utilizes NLB Hook Rules to make decisions on whether it will use the source IP address or destination IP address as affinity. Based on this we immediately checked our NLB Hook\Hash rules that TMG creates (or should create) when we created the Web Publishing rule pointing at the Internal Web Server.

* To check your Hook rules on your TMG server you will run the following command from an elevated command prompt: netsh tmg show nlb

** for more information on troubleshooting NLB review the following KB article: https://technet.microsoft.com/en-us/library/ff849728.aspx

In our test we created a Web Publishing rule and pointing to a single web server named Web.domain-01.com (The external name of the site is different though. It is External.com). The servers LAN IP address was 10.61.221.99. This is a clipping of the rule tested with:

This is what we saw when we ran the Netsh command using the “find” command to see only the IP address of that server:

It can’t find and hook rules created for that IP Address!

We deleted the Web Publishing rule and created a Server Publishing rule for the HTTP protocol and pointed the “To” tab to the same IP address:

We then ran the same Netsh command. This time we saw a much better return:

So at this point we see what looks to be a problem! We tested the Web Server rule with an internal Web Farm as well (versus the single internal server) and saw similar issues to where Hook rules were not being created.

Conclusion:

Through further testing and internal debugging of the product we were able see that we actually have multiple issues going on here. It turns out that the code behind the Web Publishing to a single server and Web Publishing to a Backend server farm behave differently and fall under separate issues. The Web Publishing to a single server is actually fixed in a hotfix that was added to Service Pack 2 for TMG. So if you see this problem when publishing a single server all you need to do is update to Service Pack 2.

“Well what was the problem?” you ask…

What is actually happening is all based off the highlighted area below:

The hook rules are created based off name resolution. The name in the Published site area is not able to be resolved via DNS internally therefore no Hook rule is created. But, when the name isn’t resolved it should default to the administratively entered IP address just below it and create the hook rule. This is not done properly prior to Service Pack 2.

* For more information on the single server issue you can refer to the following TechNet article:

FIX: Network Load Balancing hook rules are not created correctly when the name on the To tab of the Web Publishing rule does not resolve to a published web server in Forefront Threat Management Gateway 2010

https://support.microsoft.com/kb/2591269

“Can we work around this? We can’t install SP2 at this time.”

The answer to this question is yes but it is preferred that you updated to SP2. The workaround to this would be to create a hosts file on your TMG server that resolves External.com to 10.61.221.99. You would then have to recreate the rule.

I mentioned above that publishing to a backend server farm and letting TMG manage your farm instead of a 3^rd Party Load Balancing device is different than single server publishing. Due to those differences installing SP2 will not resolve your issue. You will know you are configured in this manner if you look in your rule and see the following tab:

This has been determined to be a problem and will be fixed in future versions of the product. At the time the only method of getting the bidirectional affinity hook rules in place is to utilize the following workaround:

Open the “hosts” file in notepad utilizing an elevated command prompt on the TMG Array nodes (C:\Windows\System32\drivers\etc). You will want to create an entry for the IP address of each of the servers you listed in your Web Farm in TMG In the hosts file. Those IP Addresses will resolve to the name listed in the rules “Internal site name:” setting (This would look very similar to a DNS Round Robin type of entry in DNS). Here is an example based on our rule above. We are pointing to 2 internal servers:

After saving the hosts file you will want to delete and recreate your Web Publishing rule. This will cause it to create the NLB hooks needed to have bidirectional affinity work properly therefore resolving your issue!

I hope this information helps you out!

Here is a problem statement you may be facing:

“My VPN clients are failing to connect to my environment when utilizing L2TP and connecting to the Virtual IP of the Array. If I connect to the physical IP of either of the nodes it connects without issue. PPTP and SSTP work but for some reason L2TP doesn’t. I am positive I have it setup right. What could be wrong with TMG or RRAS?”

Well, I will start by saying that if you are seeing this you may have already pulled out all your hair! There’s nothing in the event logs what-so-ever pointing to an issue of any kind! As well… when you do live tracing in TMG to see why the VPN connection is failing everything seems to be working fine. Where do you start looking for data???

Based on the issue I’m covering I’ve found that Network Monitor is a very useful tool to help troubleshoot LT2P/IPSec related issues. In addition, if you have a TMG Array with Integrated NLB enabled on the External Network, I’d suggest that you install Network Monitor on all nodes of the array as you may not be sure which node will handle the L2TP/IPSec traffic in question.

* You can download the most recent version of Network Monitor from the following location: https://www.microsoft.com/download/en/details.aspx?displaylang=en&id=4865

Once Network Monitor is installed and running, go ahead and reproduce the L2TP VPN client connection issue. Once you have reproduced your issue stop your tracing and apply the following Display Filter: IKE or ESP. This will cause Network Monitor to show you IPSec related data.

Looking through the trace you will see the following behaviors…

The client tries to setup an IPSec Tunnel with the RRAS\TMG Server’s Virtual IP Address (Keep in mind that the Microsoft L2TP protocol utilizes IPSec as an underlying security protocol). The first part of the IPSec process is what we call Main Mode. As you can see below this works fine:

The next thing we see is that the Quick Mode Process works without issue. Quick mode is the second part of the communication process the IKE protocol utilizes to setup an IPSec Tunnel:

Once Main Mode and Quick Mode have completed we will now have an IPSec tunnel created and the client will communicate through that tunnel utilizing the ESP protocol:

Now comes the odd part. Instead of the Server returning ESP communications to the Client, as it should, it tries to open a new tunnel up to the client. It does this via a new Main Mode connection over the physical IP Address on the External interface of the RRAS\TMG server instead of the Virtual IP. The client would not have a policy to support this process therefore it silently drops the requests and retransmits the ESP data it sent before. Each time the server receives the ESP data it tries to send out a Main Mode request. The behavior can be seen below:

So the question is…

“Why does the server try to create a new tunnel out to the client over the Physical IP while there is already a tunnel established over the Virtual IP address?”

Answer…

The problem is that the server is not selecting the correct IPSec Security Association (SA) when trying to communicate back. Therefore it tries to create a new one. The only way the server sees to connect up to the clients IP Address is by sending the data out of its Default Gateway which is associated with its Physical IP Address\NIC.

How do I resolve this…

This is a known issue in the Server 2008 R2 operating system. To resolve this issue all you need to do is connect to the Microsoft TechNet article below and choose the “View and request hotfix downloads” Link in the upper Left corner.

You cannot establish an IPsec tunnel to a computer that is running Windows 7 or Windows Server 2008 R2 through a NAT device

https://support.microsoft.com/default.aspx?scid=kb;EN-US;2523881

* Again… keep in mind that the data in the article only refers to IPSec and you are using L2TP. That’s perfectly ok. That’s because L2TP uses IPSec as its means of creating a secure tunnel. As well, the information covered in the article also covers NAT-T configurations. That is only 1 scenario that was found. The fix actually covers multiple scenarios although not listed.

Thank you for your time in reading this post. I hope it helps you out!