Share via

Unexpected error encountered filling record reader buffer: HadoopExecutionException: Too long string in column

M, Murugeswari (Cognizant) 456 Reputation points
2022-03-21T06:53:55.527+00:00

Hi,

We have a task to migrate tables from oracle aws to azure synapse. So initially we are loading from oracle to blob and then moving to synapse.

Out of 5, 4 tables copied successfully, but we are facing issue with one table. Getting below error:-

Unexpected error encountered filling record reader buffer: HadoopExecutionException: Too long string in column
[-1]: Actual len = [31876]. MaxLEN=[4000],},],'

185067-image.png

NOTE : We are having blob datatype for a column in the source table

Can anyone please suggest to overcome this issue

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,646 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. ShaikMaheer-MSFT 38,546 Reputation points Microsoft Employee Moderator
    2022-03-22T15:51:00.267+00:00

    Hi @Anonymous ,

    Thank you for posting query in Microsoft Q&A Platform.

    Could you please clarify where exactly you are facing this error? Is it while loading data to blob or while loading data synapse table?

    If it's while loading data to Synapse then kindly check if your sink table column length may be less than source column length. Using Sink table column data type as nvarchar(max) may help.

    If you are using Polybase option then please note, Currently using Polybase has this limitation of 1mb and length of column is greater than that. The work around is to use bulk insert in copy activity of ADF or chunk the source data into 8K columns and load into target staging table with 8K columns as well. Check this document for more details on this limitation.

    If you're using PolyBase external tables to load your tables, the defined length of the table row can't exceed 1 MB. When a row with variable-length data exceeds 1 MB, you can load the row with BCP, but not with PolyBase.

    Try using "bulk insert" option in the ADF pipeline. It may be helpful. Please let us know how it goes. Thank you.

    1 person found this answer helpful.

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.