SNI Extension Server Name in IP Address is Rejected

asked 2020-08-23T06:24:30.547+00:00
Al 1 Reputation point

Hi,

We have 2 Windows Server 2016 installation. In one installation, when a client passed an IP address to the SNI extension, the SSL exchange can proceeds. However, in another installation, if we pass an IP address, the server rejects so we can only go so far to Client Hello. All bindings have no SNI settings and both server can proceed if the SNI extension is left out.

Does anyone knows why the the other installation rejects the connection if the SNI server name has an IP address?

Thanks.

Windows Server 2016
Windows Server 2016
A Microsoft server operating system that supports enterprise-level management, data storage, applications, and communications.
1,751 questions
No comments
{count} votes

2 answers

Sort by: Most helpful
  1. answered 2020-08-24T10:11:17.597+00:00
    Carl Fan 231 Reputation points

    Hi,
    Have you tried to switch to using hostnames?
    Check if firewall turned off.
    Perform a clean boot and disable security software temporarily to check.
    https://support.microsoft.com/en-us/help/929135/how-to-perform-a-clean-boot-in-windows
    Best Regards,
    Carl

    No comments

  2. answered 2020-08-25T03:40:11.22+00:00
    Al 1 Reputation point

    Hi,

    It has nothing to do with FW because we can see from Wireshark that TCP connection can go through. In the installation where the server rejects the IP address in the SNI, it issues a tcp RST,ACK after Client Hello. But if I pass a FQDN in the SNI, the SSL exchange completes.

    I understand that passing a literal IP address in the SNI is a violation of SSL protocol so rejecting this connection maybe the right behavior . What I don't understand is why a different server with the same version of Windows Server would allow this.

    Regards.

    Al

    No comments