How to connect to NetSuit database from Azure synapse spark notebook using pyspark ?

heta desai 252

I want to connect to the NetSuit database from Azure synapse spark notebook using pyspark. While using the below command it throws an error.

jdbcDF = spark.read.format("cdata.jdbc.netsuite.NetSuiteDriver") \
    .option("url", url) \
    .option("user", user) \
    .option("password", password) \
    .option("query",query) \
    .load()

Error: Py4JJavaError: An error occurred while calling o649.load.
: java.lang.ClassNotFoundException: Failed to find data source: cdata.jdbc.netsuite.NetSuiteDriver. Please find packages at http://spark.apache.org/third-party-projects.html

PRADEEPCHEEKATLA 90,246 Reputation points

2022-05-04T07:14:28.557+00:00

Hello @Anonymous ,

Thanks for the question and using MS Q&A platform.

From the error message - it clearly says the cdata.jdbc.netsuite.NetSuiteDriver does not exists on the Spark Runtime?

Make sure you have download the required jar file and upload the jar file to the workspace packages.

For more details, refer to Manage Scala and Java packages for Apache Spark in Azure Synapse Analytics and Process & Analyze NetSuite Data in Azure Databricks.

Hope this helps.
heta desai 252 Reputation points

2022-05-04T11:17:31.067+00:00

It asks for configuration file to install the package. How to create that configuration file to install the package into spark pool of Azure Synapse.
PRADEEPCHEEKATLA 90,246 Reputation points

2022-05-05T07:18:25.997+00:00

Hello @Anonymous ,

As I said above, you can download the required jar file and upload the jar file to the workspace packages.

Share via

How to connect to NetSuit database from Azure synapse spark notebook using pyspark ?

Your answer