SSIS package row delimiter Failure
Hi,
I have a package that reads a file from a network location and inserts the data into a database. The data flow task includes four components:
- Flat File Source
- Derived Column to convert
DT_NText
toDT_WTSR
- Derived Column to convert
DT_WTSR
toDT_STR
- OLE DB Destination
The flat file source reads a delimited text file with a header row formatted as {CR}{LF}
and a text qualifier of "
.
I've added an error flow to log problematic data and process the remaining rows.
There is some bad data in the file I'm trying to load. Below is a sample of the data; the actual file contains millions of records:
"ClientID","SubscriberKey","EmailAddress","SubscriberID","Status","DateHeld","DateCreated","DateUnsubscribed"
10416502," ******@yahoo.com"," ******@yahoo.com ",343044425,"Bounced",,4/10/2019 1:01:00 PM,
10416502,""******@wellsfargo.com
"",""******@wellsfargo.com
"",343044426,"Undeliverable",4/10/2019 1:01:00 PM,4/10/2019 1:01:00 PM,
10416502,"@msjnet.net","@msjnet.net",343044427,"Active",,4/10/2019 1:01:00 PM,
The expected outcome is to skip the bad data and process the remaining records; however, the package is failing at the flat file source component with the following error.
Error Description: The column delimiter for column "SubscriberKey" was not found.
Error Description: An error occurred while processing file "Subscribers.txt" on data row 3.
Any suggestions or recommendations on how to handle this situation?
Flow:
Dataflow_task.png