Validation Activity - Not giving right error

Kothai Ramanathan 941 Reputation points Microsoft Employee
2021-02-19T08:57:44.177+00:00

I have a pipeline which works on files in a folder. At the start of the pipeline is a validate activity to check if files exist. If there are no files, then we do nothing. Because we do not want the pipeline to report an error every time there are no files. As files not being present in a valid business scenario.

However notice that if there are other failures in the validation activity (either the path provided is not right, or the credentials provided are not right), even then it only fails the activity, but not give the error for which it failed.

69909-image.png

In the above scenario, I have provided a wrong directory name, and the activity failed, but there is no error that the parameter is wrong. If I do the same thing to a copy activity, it rightly provides an error mentioning that the parameter is wrong.

Because of this, we are not able to identity whether the activity failed because of files not found, or there was real issue when accessing the files. How do we differentiate such cases ?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,904 questions
{count} votes