Server Sent RST , ACK immediate after received Client Hello

subra 1 Reputation point
2020-04-07T11:10:13.66+00:00

my webserver unable to handshake with A10 Load Balancer.
as traced through wire shark, the connection from A10 LB getting reset by my webserver immediately after received Client Hello from A10 LB. both end the TLS 1.2 enabled and already set the required Cipher suites. even I already used NARTAC software to apply the recommended TLS and Ciphers setting.

but issue still not resolved. below two error I found in event viewer. please help to guide me to resolve this issue

Event ID : 36874 - An TLS 1.2 connection request was received from a remote client application, but none of the cipher suites supported by the client application are supported by the server. The SSL connection request has failed.

Event ID : 36888 - A fatal alert was generated and sent to the remote endpoint. This may result in termination of the connection. The TLS protocol defined fatal error code is 40. The Windows SChannel error state is 1205.

Wireshark Trace :

Frame 1715: 54 bytes on wire (432 bits), 54 bytes captured (432 bits) on interface \Device\NPF_{AF37DEDF-E8F9-475A-B504-8FFCE3B723D3}, id 0
Ethernet II, Src: Vmware_9c:21:ad (00:50:56:9c:21:ad), Dst: MS-NLB-PhysServer-31_a0:00:00:09 (02:1f:a0:00:00:09)
Internet Protocol Version 4, Src: XXX.XXX.XXX.39, Dst: XXX.XXX.XXX.79
Transmission Control Protocol, Src Port: 443, Dst Port: 13446, Seq: 1, Ack: 115, Len: 0
Source Port: 443
Destination Port: 13446
[Stream index: 27]
[TCP Segment Len: 0]
Sequence number: 1 (relative sequence number)
Sequence number (raw): 1957420587
[Next sequence number: 1 (relative sequence number)]
Acknowledgment number: 115 (relative ack number)
Acknowledgment number (raw): 1333508135
0101 .... = Header Length: 20 bytes (5)
Flags: 0x014 (RST, ACK)
000. .... .... = Reserved: Not set
...0 .... .... = Nonce: Not set
.... 0... .... = Congestion Window Reduced (CWR): Not set
.... .0.. .... = ECN-Echo: Not set
.... ..0. .... = Urgent: Not set
.... ...1 .... = Acknowledgment: Set
.... .... 0... = Push: Not set
.... .... .1.. = Reset: Set
[Expert Info (Warning/Sequence): Connection reset (RST)]
[Connection reset (RST)]
[Severity level: Warning]
[Group: Sequence]
.... .... ..0. = Syn: Not set
.... .... ...0 = Fin: Not set
[TCP Flags: ·······A·R··]
Window size value: 0
[Calculated window size: 0]
[Window size scaling factor: 256]
Checksum: 0x5792 [unverified]
[Checksum Status: Unverified]
Urgent pointer: 0
[SEQ/ACK analysis]
[This is an ACK to the segment in frame: 1714]
[The RTT to ACK the segment was: 0.002502000 seconds]
[iRTT: 0.000587000 seconds]
[Timestamps]
[Time since first frame in this TCP stream: 0.003095000 seconds]
[Time since previous frame in this TCP stream: 0.002502000 seconds]

Not Monitored
Not Monitored
Tag not monitored by Microsoft.
35,882 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Dave Patrick 426K Reputation points MVP
    2020-04-07T16:15:38.033+00:00

    QnA currently supports the products listed in right-hand pane (more to be added) Better to reach out to subject matter experts in dedicated forums over here.

    https://forums.asp.net/

    https://forums.iis.net/

    (please don't forget to mark helpful replies as answer)

    0 comments No comments