There are several steps to make it work.
Let's assume you have SERVER1 on which you want to restrict the access to port 80 to users member of a specific AD group called CONTOSO\Access80.
On SERVER1, you need to do two things. All is happening in the Firewall console (that you can launch with the shortcut
- Set the Inbound rule of the firewall to Allow connection if it is secure
Then click to Customize and click Allow the connection to use null encapsulation.
This option will essentially not encrypt the data. The reason why I suggest this option is that you don't say you need encryption, you say you need to authenticate and restrict. With this option, the traffic will still show up as usual hence facilitating the network troubleshooting if you have issues (which might be the case if you are new to IPSec - because yes, all we do here is configure IPSec).
Then on the Remote Users tab, check Only allow connections from these users and pick your group from the picker.
- Now, still on SERVER1 you need a Connection Security Rule. Still on your Firewall console, right click on Connection Security Rules and click New.
In the first screen, select Custom (the wizard is hiding a complexity which I don't believe exist... So we go custom right away as it often makes it simplier at the end).
In the next screen, you can leave everything by default.
Then pick the second option Require authentication for inbound connections and request authentication for outbound connections.
Then leave the default option in the Authentication Method.
In the Protocol and Ports, pick TCP, on Enpoint 1 port select Specified Ports and 80. Don't change anything on Endpoint 2 port.
Leave all Profile enabled, click Next and give it a name before saving.
Then on the clients used by your users (and you can do that directly through the Firewall console or your can deploy that through group policy), you need to do only one thing:
- Create a Connection Security Rule. The equivalent of the opposite rule we created on the SERVER1.
You pick Custom again.
In Endpoints you can specify the IP address of SERVER1 in the Which computers in Endpoint 2 section.
In Requirements you can pick the first option Request authenication for inbound and outbound connections.
Leave the Default option in the Authentication Method.
In Protocol and Ports use the reverse config such as:
Leave all profile on and give it a name before saving.
Now you server will require IPSec authentication for the inbound port 80 and your clients will try to do IPSec authentication when they connect to the port 80 of the SERVER1 (because of its IP address). And for the connection to work, SERVER1 will enforce the group membership you have set in the inbound rule.