MS Spark SQL Connector AD Integration Error

Dinesh Balachandran 56 Reputation points
2023-02-27T17:02:21.53+00:00

We have been using Azure Synapse notebooks to connect and load some data from Azure SQL. We were using the SQL Authentication Model and were looking to use the Azure SQL Spark Connector to use AD

While we were able to use the examples to get the code working via Service Principal Token model. We get the following error when trying to use the AD Password mode.

Py4JJavaError: An error occurred while calling o2293.load. : java.lang.NoClassDefFoundError: com/nimbusds/oauth2/sdk/auth/ClientAuthentication

Any inputs are greatly appreciated

I had posted the snapshot of the code in the comments

jdbc_df = spark.read.format("com.microsoft.sqlserver.jdbc.spark").option("url", "jdbc:sqlserver://Z-ondemand.sql.azuresynapse.net") \
.option("dbtable", "dbo.MyTable").option("authentication", "ActiveDirectoryPassword") \
.option("user","a@test.com").option("password", "XXXXXXXXXXXX").option("encrypt", "true")\
.option("hostNameInCertificate", "*.database.windows.net")\
.option("driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver")\
.load()
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,447 questions
{count} vote