Azure Databricks SSHTunnelForwarder error Could not open connection to gateway

Question

This problem started to happen since December 24, 2021. There was no problem before that date. My Spark run-time is 8.4 (Spark 3.1.2 and Scala 2.12). Running the same codes outside Azure Databricks has no problem so all SSH tunnel credentials are correct and working. My test script is shown below. Basically, I tried to open a SSH tunnel to connect to a MySQL database hosted in AWS. The error points to the "SSHTunnelForwarder" command with error "Could not open connection to gateway ". Why are the same codes working outside Azure Databricks, but failing in Azure Databricks?

import pymysql # https://pypi.org/project/PyMySQL/
import paramiko # https://pypi.org/project/paramiko/
import pandas as pd
from paramiko import SSHClient
from sshtunnel import SSHTunnelForwarder # https://pypi.org/project/sshtunnel/

mypkey = paramiko.RSAKey.from_private_key_file(<omitted_for_security>)

sql_hostname = <omitted_for_security>
sql_username = <omitted_for_security>
sql_password = <omitted_for_security>
sql_database = <omitted_for_security>
sql_port = 3306
ssh_host = <omitted_for_security>
ssh_user = <omitted_for_security>
ssh_port = 22

with SSHTunnelForwarder(
(ssh_host, ssh_port),
ssh_username=ssh_user,
ssh_pkey=mypkey,
remote_bind_address=(sql_hostname, sql_port)) as tunnel:
conn = pymysql.connect(host='127.0.0.1', user=sql_username,
passwd=sql_password, db=sql_database,
port=tunnel.local_bind_port)
query = '''SELECT VERSION();'''
data = pd.read_sql_query(query, conn)
display(data)
conn.close()

---

BaseSSHTunnelForwarderError Traceback (most recent call last)
<command-1233393883717395> in <module>
24
---> 25 with SSHTunnelForwarder(
26 (ssh_host, ssh_port),
27 ssh_username=ssh_user,

/databricks/python/lib/python3.8/site-packages/sshtunnel.py in enter(self)
1606 def enter(self):
1607 try:
-> 1608 self.start()
1609 return self
1610 except KeyboardInterrupt:

/databricks/python/lib/python3.8/site-packages/sshtunnel.py in start(self)
1329 self._create_tunnels()
1330 if not self.is_active:
-> 1331 self._raise(BaseSSHTunnelForwarderError,
1332 reason='Could not establish session to SSH gateway')
1333 for _srv in self._server_list:

/databricks/python/lib/python3.8/site-packages/sshtunnel.py in _raise(self, exception, reason)
1172 def _raise(self, exception=BaseSSHTunnelForwarderError, reason=None):
1173 if self._raise_fwd_exc:
-> 1174 raise exception(reason)
1175 else:
1176 self.logger.error(repr(exception(reason)))

BaseSSHTunnelForwarderError: Could not establish session to SSH gateway

Answer 1

Hello @Joseph Chen ,

Here is the summary of the support case:

You encountered an issue with Azure Databricks SSHTunnelForwarder error Could not open connection to gateway

Root Cause Analysis: We have found a change in the paramiko version (host key algorithm) which was the reason of the issue.

Solution: Installing the old version of paramiko (2.8.1 , command: %pip install paramiko==2.8.1) worked and resolved the issue.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Answer 2

Add additional details here.

We have found a change in the paramiko version (host key algorithm) which caused the issue.
https://www.paramiko.org/changelog.html says:

2.9.0 version was introduced on 23.12.2021 (since 24th December 2021 the issue started happening) and below changes were implemented:
2.9.0 2021-12-23
• [Feature] #1643: (also #1925, #1644, #1326) Add support for SHA-2 variants of RSA key verification algorithms (as described in RFC 8332) as well as limited SSH extension negotiation (RFC 8308).