Dedicated pool table data type String or binary data would be truncated.

Rohit Kulkarni 676

Hello Team,

I am trying to load the data from parquet file to Dedicated sql pool tables with below mentioned datatype :

I am facing the issue in description column

The data contain in this particular column is :

I am getting error as :

Unexpected error encountered filling record reader buffer: HadoopSqlException: String or binary data would be truncated. [ErrorCode = 107090] [SQLState = S0001]

But if i remove particular column from the table then it is working fine.

Please advise

1 answer

Suba Balaji 11,186 Reputation points

2022-10-04T06:10:10.453+00:00

Hi @Rohit Kulkarni ,

The error usually occurs when the data is too large to fit in the sink column

In your case, if your description column brings 2000 characters and the sink table allows only 1500 characters for description column, then while the data is inserted to table, it throws above error

To fix the same, kindly increase the length of the description column (nvarchar(max) may be) in sink table and rerun the pipeline/notebook

Let us know how it goes.

Thanks
Please sign in to rate this answer.
Rohit Kulkarni 676 Reputation points

2022-10-04T07:49:35.583+00:00

Hello Subashri,

I am passing the dayatype for description in this format StructField("ActualCost", DoubleType(), True) .As per above comments how i can pass nvarchar(max) in the same format.

Regards
Rohit

Rohit Kulkarni 676 Reputation points

2022-10-04T08:05:13.3+00:00

Please Ignore my previous messages

Hello Subashri,

I am passing the dayatype for description in this format StructField("Description", Stringtype(), True) .As per above comments how i can pass nvarchar(max) in the same format.

Regards
Rohit

Rohit Kulkarni 676 Reputation points

2022-10-04T17:26:27.217+00:00

If you refer the data in the description column it is mentioned in the below format :

Continuation of this meeting series in 2021
Hi
This is the invitation to the sprint review (https://wikipedia.org/wiki/Scrum\_(software_development)#Sprint_review) of software activities in projects
* Net
* Link
* Optima
* lus
The goal of this meeting is providing a regular status update and – more importantly- give you a chance to suggest improvement.
Attendance is voluntary.

Join Microsoft Teams Meeting<https://teams.microsoft.com/l/meetup-join/19%3ameeting_M2Q5ZGI0MzYtNzc5ZS00NTk3LWJlNzMtMjg3M2JlNDZjOGYw%40thread.v2/0?context=%7b%22Tid%22%3a%22f98eb369-7c22-4e6f-947f-a5d05fb884c0%22%2c%22Oid%22%3a%22c17a7cea>
Learn more about Teams<https://aka.ms/JoinTeamsMeeting> | Meeting options<https://teams.microsoft.com/meetingOptions/?organizerId=c17a7cea-b836-4a88-a62f-ece4c38be8ca&tenantId=f98eb369-7c22-4e6f-947f-a5d05fb884c0&threadId>

And even the datatype in the synapse analytics for this particular decription column is nvarchar(4000)

Please advise
Sign in to comment