Share via

accessing Synapse Table from databricks using pyspark

Naveen Kumar 1 Reputation point
2022-03-22T07:27:50.957+00:00

sql_pwd= dbutils.secrets.get("keys and scope")
2
urrl="jdbc:sqlserver://synapsewus2prod.sql.azuresynapse.net:1433;database=<db_name>;user=<user_name>;password="+sql_pwd+";encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.sql.azuresynapse.net;loginTimeout=30;"
3
df=spark.read.format("com.databricks.spark.sqldw").option("useAzureMSI","true").option("url", urrl).option("dbtable", "table name").option("tempDir","wasbs://<container location>").load()

tried the provided solution but it didn't helped. Throwed an error saying, "
com.databricks.spark.sqldw.SqlDWSideException: Azure Synapse Analytics failed to execute the JDBC query produced by the connector."

could you please help, I am trying to Update a synapse table from the pyspark dataframe that I have in Databricks.
I am having a synapse table with columns A,B,C,D and I have a Dataframe that has the values for column D that I need to update in the Synapse Table.

Azure SQL Database
Azure Synapse Analytics
Azure Synapse Analytics

An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.

Azure Databricks
Azure Databricks

An Apache Spark-based analytics platform optimized for Azure.


Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.