SNI Extension Server Name in IP Address is Rejected

Al 21 Reputation points
2020-08-23T06:24:30.547+00:00

Hi,

We have 2 Windows Server 2016 installation. In one installation, when a client passed an IP address to the SNI extension, the SSL exchange can proceeds. However, in another installation, if we pass an IP address, the server rejects so we can only go so far to Client Hello. All bindings have no SNI settings and both server can proceed if the SNI extension is left out.

Does anyone knows why the the other installation rejects the connection if the SNI server name has an IP address?

Thanks.

Windows Server 2016
Windows Server 2016
A Microsoft server operating system that supports enterprise-level management updated to data storage.
2,371 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Carl Fan 241 Reputation points
    2020-08-24T10:11:17.597+00:00

    Hi,
    Have you tried to switch to using hostnames?
    Check if firewall turned off.
    Perform a clean boot and disable security software temporarily to check.
    https://support.microsoft.com/en-us/help/929135/how-to-perform-a-clean-boot-in-windows
    Best Regards,
    Carl

    0 comments
  2. Al 21 Reputation points
    2020-08-25T03:40:11.22+00:00

    Hi,

    It has nothing to do with FW because we can see from Wireshark that TCP connection can go through. In the installation where the server rejects the IP address in the SNI, it issues a tcp RST,ACK after Client Hello. But if I pass a FQDN in the SNI, the SSL exchange completes.

    I understand that passing a literal IP address in the SNI is a violation of SSL protocol so rejecting this connection maybe the right behavior . What I don't understand is why a different server with the same version of Windows Server would allow this.

    Regards.

    Al

    0 comments