TDS-Protocol with pre-login failure message when connecting to Amazon RDS (MS SQL)

Margas, Marek (FDF2) 5 Reputation points
2023-07-05T05:51:18.2766667+00:00

We are trying to connect from an application installed on a virtual machine to a database as part of an AWS RDS in the cloud.

There are several routers and firewalls between the source and the target system and there is also NATting happening of the source IP.

A connection test in PowerShell works.

Connection Test in PowerShell_NEU

However, when trying to connect to the database with a connection string from within the application, a TDS pre-login handshake error message appears.

Handshake error TDS

A traffic capture with Wireshark also did not provide a clear solution to the error.

We have also tested the connection to the database from SSMS (SQL Server Management Studio) and the same error message appears.

When testing the connection from a different virtual machine the problem does not occur.

So it is not due to a specific application.

My guess is that somehow the SQL Server can't understand the pre-login message and kills the connection.

Does anybody know how to analze this pre-login message in Wireshark to derive further possible solutions?

We turned off the encryption on the AWS RDS but this did not help.

SQL Server
SQL Server
A family of Microsoft relational database management and analysis systems for e-commerce, line-of-business, and data warehousing solutions.
14,289 questions
Not Monitored
Not Monitored
Tag not monitored by Microsoft.
40,946 questions
0 comments No comments
{count} votes

2 answers

Sort by: Most helpful
  1. Margas, Marek (FDF2) 5 Reputation points
    2023-07-12T05:32:12.8733333+00:00

    I just wanted to give feedback that the problem has been resolved.

    It was indeed due to a firewall activation that was not implemented correctly a few weeks ago.

    1 person found this answer helpful.

  2. Erland Sommarskog 115.3K Reputation points MVP
    2023-07-05T12:20:46.34+00:00

    As I understand the screenshot with PowerShell, you are only testing the TCP connection as such. That is, there is no actual TDS traffic occurring. And thus, there can not be any TDS errors.

    I can think of two things:

    1. Increase the connection timeout. You can do this in the connection string. You can also do this on the Connection page in SSMS.
    2. Check the SQL Server errorlog for errors that may be related to the login failures.

    Are you using SQL authentication or Windows authentications?

    As this may be something that is RDS-specific, you may have better luck in an Amazon forum.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.