Problem with ADF Dataflow pipeline when writing data to SQL Server

Smagulov, Alisher [Norsec Delta Projects LLP] 0

I have an ADF Dataflow pipeline that reads data from a data lake and writes it to a SQL Server table. I have a problematic column called "gdtxft" that I converted from binary to string using the 'toString' function in the Derived column setting. The column has a size of nvarchar(max) in SQL Server.

The problem is that when ADF Dataflow writes to SQL Server, it only writes the first character in the string and when I save the results using Excel, I can see all the string characters. I have verified that the data is being stored correctly in SQL Server by running queries on the table.

I have also checked the pipeline settings and verified that the correct encoding is being used and that the data is being read and written correctly. I have tried reading the data from the source using other tools and comparing the results with the data in SQL Server, but I haven't found any differences.

I would appreciate any help or suggestions on how to resolve this issue.

Thank you.

Please let me know if there's anything else I can User's image

help you with. User's image

User's image

Subashri Vasudevan 11,216 Reputation points

2023-01-26T05:30:54.99+00:00

Hi, could you please check what's the length of that field in the sink dataset schema tab ?? Or else pls post the schema screenshot and derived column expression too
Subashri Vasudevan 11,216 Reputation points

2023-01-27T15:55:32.6633333+00:00

Hi, just checking if you still require assistance here.
Bhargava-MSFT 31,196 Reputation points Microsoft Employee

2023-01-30T22:15:23.6466667+00:00

Thank you, @Subashri Vasdudevan

Hello @Smagulov, Alisher [Norsec Delta Projects LLP]

I am checking to see if you need further assistance here.

Share via

Problem with ADF Dataflow pipeline when writing data to SQL Server

Your answer