Data insert failing to SQLMI from Databricks runtime 9.1LTS

Question

Hi,

I am inserting data into sqlserver table using a bulk copy per the below code.
%scala
import com.microsoft.azure.sqldb.spark.bulkcopy.BulkCopyMetadata
import com.microsoft.azure.sqldb.spark.config.Config
import com.microsoft.azure.sqldb.spark.connect._

val Datawriter = spark.sql("select * from testtable")  
  
val bulkCopyConfig = Config(Map(  
                              "url"             -> jdbcHost,  
                              "databaseName"    -> jdbcDatabase,  
                              "user"            -> jdbcUsername,  
                              "password" -> jdbcPassword,  
                              "dbTable" -> jdbcTableName,  
                              "bulkCopyBatchSize" -> "2000",  
                              "bulkCopyTablelock" -> "true",  
                              "bulkCopyTimeout" -> "6000"))  
  
Datawriter.bulkCopyToSqlDB(bulkCopyConfig)  

the above is throwing error as below
NoSuchMethodError: com.microsoft.sqlserver.jdbc.SQLServerBulkCopy.writeToServer(Lcom/microsoft/sqlserver/jdbc/ISQLServerBulkRecord;)V

as per google and other forums i need to install mssql-jdbc library, but there are a number of versions of that library and i am unable to figure out which one is relevant to the cluster configuration.
I have tried installing quite a few of : com.microsoft.sqlserver:mssql-jdbc:

the cluster configuration is in the attached screenshot.

my scala code with error is also attached in the screenshot

can someone please help with this.

Answer 1

@PRADEEPCHEEKATLA I dont have access to support plan. Can you please lodge a ticket with a high priority.
Thanks

Answer 2

@AHMAD Muhammad , hi I got the same errors as you. Did you find any solution to this?
Thanks in advance