Azure Data Factory - Cannot detect a row after parse 100 MB data.,Source=Microsoft.DataTransfer.Common,'

Question

I am running a Data Factory process to load Delimited File (1.2 GB) into Azure SQL server and it's failing after the Data read reached to 100 mb.

Error:

Error
Operation on target ForEach1 failed: Activity failed because an inner activity failed; Inner activity name: Copy data1, Error: ErrorCode=DelimitedTextIncorrectRowDelimiter,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The specified row delimiter
is incorrect. Cannot detect a row after parse 100 MB data.,Source=Microsoft.DataTransfer.Common,'

And the failure point is not consisted first time it failed around 17 million records, second time around 19 million records so it's not consistent to validate the file.

Any help is appreciated.

Answer 1

Hello @Venkat Balla ,
Thanks for the ask and using the Microsoft Q&A platform .

There is a possiblity that the row size is bigger then 100MB ( default) and so you are hitting this error . Can you please introduce the property "MaxRowSize" in the dataset and see if that helps.
"MaxRowSize": 2000000
The below animation should help in setting this property .

Please do let me know how it goes .
Thanks
Himanshu
Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

Answer 2

Thank you. I tried that but it still failed but with a slightly different error. 1 MB instead of 100 MB.

Updated Property:

        "columnDelimiter": ",",
        "escapeChar": "\\",
        "firstRowAsHeader": true,
        "quoteChar": "\"",
        "MaxRowSize": 2000000

Error:

ErrorCode=DelimitedTextIncorrectRowDelimiter,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The specified row delimiter is incorrect. Cannot detect a row after parse 1 MB data.,Source=Microsoft.DataTransfer.Common,

Answer 3

Sorry for delayed response. In my case issue is with file format once we corrected the file format we were able to resolve this issue.

Answer 4

Hi Were you able to find any resolution for this error to load a file greater than 100mb

thanks

Answer 5

My file size is 3gb, I have a similar issue. I tried changing the "MaxRowSize": 900000000, I get an error message

ErrorCode=DelimitedTextIncorrectRowDelimiter,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The specified row delimiter  is incorrect. Cannot detect a row after parse 858 MB data.,Source=Microsoft.DataTransfer.Common,'

Can someone please help?