Notebook writes data to the Azure database appears The connection is closed

夏辉 26

It's been going on for days and it hasn't gotten any better. The database also could not find a restart button
There are no problems with reading.

python :

usersdf.write \
.format("com.microsoft.sqlserver.jdbc.spark") \
.mode("append") \
.option("schemaCheckEnabled", False) \
.option("url", "jdbc:sqlserver://xxxx.database.windows.net:1433;database=xxxx") \
.option("dbtable", "xxxx") \
.option("user", "xxxxx") \
.option("password", "xxxx") \
.save()

error:

Py4JJavaError Traceback (most recent call last)

PRADEEPCHEEKATLA 90,446 Reputation points

2022-09-13T11:55:43.083+00:00

Hello @夏辉 ,

Thanks for the question and using MS Q&A platform.

This could be an intermettitent issue. Could you please re-try and do let us know the status?
夏辉 26 Reputation points

2022-09-14T01:18:01.363+00:00

Unbelievable, it's fine on its own。

The problem continued from September 8th to September 13th.

Can you tell me why, I need to explain it to the customer
PRADEEPCHEEKATLA 90,446 Reputation points

2022-09-14T05:39:24.453+00:00

Hello @夏辉 ,

This issue looks strange. For a deeper investigation and immediate assistance on this issue, if you have a support plan you may file a support ticket

We are not sure on what could be the actual reason for the failure: This could be the (network, cluster, azure database, or ADB cluster outages).

The Azure Databricks Status Page provides an overview of all core Azure Databricks services. You can easily view the status of a specific service by viewing the status page. Optionally, you can also subscribe to status updates on individual service components, which sends an alert whenever the status you are subscribed to changes.
PRADEEPCHEEKATLA 90,446 Reputation points

2022-09-19T07:05:39.73+00:00

Hello @夏辉 ,

Did you get a chance to open a support ticket?
夏辉 26 Reputation points

2022-09-20T03:04:29.43+00:00

Thanks, the problem has been solved

1 answer

夏辉 26 Reputation points

2022-09-20T03:03:41.603+00:00

The problem has been solved
The dataframe and DB data types are inconsistent
Please sign in to rate this answer.

1 person found this answer helpful.
PRADEEPCHEEKATLA 90,446 Reputation points

2022-10-10T06:47:26.07+00:00

Hello @夏辉 ,

Glad to know your issue has been resolved.

Sirilakshmi Veerapaneni 1 Reputation point

2023-06-14T18:18:34.79+00:00

Hi, I am facing the same error from databricks notebook which was running fine . I have compared the data types of Dataframe and Database table. They are compatible and null values are also compatible. any one know, what was the solution for this error?

my write statement is as follows: I have tried all the mssqlIsolationlevel Options.But no luck. Any early answers are much appreciated.Thanks.

url="jdbc:sqlserver://{0}:{1};database={2}".format(HostName,Port,Database); df.write \ .format("com.microsoft.sqlserver.jdbc.spark") \ .mode("overwrite") \ .option("url", url) \ .option("databaseName", Database) \ .option("dbtable", sink_table) \ .option("user", user) \ .option("password", password) \ .option("schemaCheckEnabled", False) \ .option("BatchSize", 2000000) \ .option("mssqlIsolationLevel", "READ_COMMITTED") \ .option("reliabilityLevel","BEST_EFFORT") \ .option("tableLock","true") \ .option("truncate","true") \ .save()
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Notebook writes data to the Azure database appears The connection is closed

1 answer

Your answer