Azure Data Factory pipeline fails with error code 2200.

Question

I have created a ADF pipeline which consists of a trigger associated with Storage account of type "BlobStorage." The Trigger triggers when a blob is uploaded to the storage account and the pipeline copies the data from the storage account to azure data explorer (Kusto). When I upload a folder consisting of 7 files (consisting of 64 rows each), the pipeline is triggered for all the 7 files (as expected) but all the 7 pipeline run fails with the reason listed in image, however I am able to see all 64 rows from one file in the sink kusto table (and not from other 6 files), but the pipeline fails for all the 7 files.

The "ThreadID" column (type: string) has '' (empty) value for some rows in source and in the sink kusto table the "ThreadID" column type is long. I read articles saying the string conversion are done by ADF for you and we don't have to worry.

While creating the trigger the ADF portal says that the trigger works only for the "general purpose v2 storage" account types, our storage account is of type "BlobStorage". Could this be the reason of failure?

I tried created an exact replica of the pipeline and trigger and connected it with a "general purpose v2 storage" account type. When I upload the same folder consisting of 7 files, the pipeline gets triggered for all the 7 files and all the 7 pipeline runs are successful and all the data from all the files is copied to kusto table.

I don't want to upgrade my storage account to "general purpose v2 storage", how can I get my pipeline working by getting rid of the error stated.

Thanks :)

Answer 1

Hello @BhagyeshPatil-0681,

Thanks for the question and using MS Q&A platform.

Are you quoting your texts in the transformation :

Looking at the error,I am assuming that you null threadID in the source.

When Quote text is enable, it trying to convert the string '' - which is not null ?

If this is your case, can you please uncheck the option and give it a try ?