Validation Activity - Timeouts event when a file is available

Kothai Ramanathan 946 Reputation points Microsoft Employee
2020-08-20T07:48:07.457+00:00

I have a validation activity which is mapped to file folder dataset. The childitems is set to true and timeout to 10 seconds. It has both a success activity and a failure activity.

  1. Now though the validate activity output is showing the file in the folder as child items, the failure activity is getting fired. Why is this ?
  2. though the timeout is set to 10 secs, it it timing out in about a minute (though in this case it should not have timed out).

Any pointers will be helpful, thanks
19009-image.png

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
{count} votes

Accepted answer
  1. HarithaMaddi-MSFT 10,146 Reputation points
    2020-08-20T14:11:44.65+00:00

    Hi @Kothai Ramanathan ,

    Welcome to Microsoft Q&A Platform.

    1) Now though the validate activity output is showing the file in the folder as child items, the failure activity is getting fired. Why is this ?

    Increase the timeout as 2seconds is less for the activity to fetch the file, please increase it to 2minutes as it is seems to take ~1minute to fetch in your scenario and check. It worked for me after doing the same as shown below.

    2) though the timeout is set to 10 secs, it it timing out in about a minute (though in this case it should not have timed out).

    Yes, able to reproduce this behavior as well and I observed that if timeout period is less than the fetching time from the blob, it will time out only after the fetching time. Also, there is slight variation in timeout Ex: when I set timeout as 10sec it timed out after 12sec and when I set timeout as 5sec it timed out after 8sec, I am checking with internal teams to confirm if this delay is because it includes the reach out time to blob as additional to 10seconds. I will confirm you once I hear back from them.

    Hope this helps! Please let us know if scenario understanding is different or for more queries and we will be glad to assist.

    19148-validationactivityoutputadf.png


0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.