The error you're encountering is due to a data type mismatch. The system expects a NVARCHAR(10) data type, but it seems that the data being read does not match this specification. Please Check if there are any special characters or data that exceed the expected length of NVARCHAR(10) in the source data. or If the schema is correct, consider using a CAST or CONVERT function in your query to explicitly convert the data to the expected type.

testing reading from dedicated pool error

Jon Z 40

Following

Bhargava-MSFT 29,266 Reputation points Microsoft Employee

2024-05-22T17:07:26.25+00:00

Hello Jon Z,

We can't see the error message. Could you please provide more details regarding your issue

Jon Z 40

import com.microsoft.spark.sqlanalytics
from com.microsoft.spark.sqlanalytics.Constants import Constants

df1 = (spark.read
.option(Constants.DATABASE, "XXX")
.option(Constants.SERVER, "yyy.sql.azuresynapse.net")
.option(Constants.QUERY, "select * from zzz.aaa1")
.synapsesql()
)

df1.show()

Jon Z 40 Reputation points

2024-05-23T08:24:54.7933333+00:00

this the code just for testing purposes

1 answer

Bhargava-MSFT 29,266 Reputation points Microsoft Employee

2024-05-23T20:55:33.52+00:00

The error you're encountering is due to a data type mismatch. The system expects a NVARCHAR(10) data type, but it seems that the data being read does not match this specification.

Please Check if there are any special characters or data that exceed the expected length of NVARCHAR(10) in the source data.

or If the schema is correct, consider using a CAST or CONVERT function in your query to explicitly convert the data to the expected type.
Please sign in to rate this answer.

1 person found this answer helpful.
Jon Z 40 Reputation points

2024-05-23T08:24:08.5166667+00:00

Py4JJavaError: An error occurred while calling o4050.showString. : com.microsoft.spark.sqlanalytics.SQLAnalyticsConnectorException: com.microsoft.sqlserver.jdbc.SQLServerException: Query aborted-- the maximum reject threshold (0 rows) was reached while reading from an external source: 1 rows rejected out of total 4735 rows processed. Column ordinal: 0, Expected data type: NVARCHAR(10) collate SQL_Latin1_General_CP1_CI_AS NOT NULL.

Jon Z 40 Reputation points

2024-05-23T08:25:15.7+00:00

this the first lines of error returned

Jon Z 40 Reputation points

2024-05-24T07:27:55.1633333+00:00

thanks for the reply ... regarding your answer, why is "show" is expecting nvarchar(10) ?

Bhargava-MSFT 29,266 Reputation points Microsoft Employee

2024-05-24T21:02:28.2233333+00:00

When you call show(), Spark reads the data from the DataFrame and tries to display it in a format that is easy to read. However, if the data in the DataFrame does not match the expected data types, Spark may encounter errors when trying to display the data.

In this case, it's possible that the data in the first column of the result set is not actually an NVARCHAR(10) data type, or that there is a null value or other data issue that is causing the error. You may need to investigate the data in the Synapse Analytics table and the query you're using to retrieve the data to determine the root cause of the issue.

Once you have identified the issue, you can modify the query or the DataFrame schema to ensure that the data types match and that there are no unexpected null values or other data issues.

Bhargava-MSFT 29,266 Reputation points Microsoft Employee

2024-05-30T15:51:53.2166667+00:00

I am checking to see if you have any further question here.

Jon Z 40 Reputation points

2024-05-30T16:26:31.1166667+00:00

thanks for you help, it is enough for me by now

Bhargava-MSFT 29,266 Reputation points Microsoft Employee

2024-05-30T17:37:19.2166667+00:00

Thank you Jon Z

If you don't have any further questions, please consider accepting the answer by hitting the Accept answer and up-vote as it helps the community look for answers to similar questions.
Sign in to comment

Share via

testing reading from dedicated pool error

1 answer