Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,196 questions
run time of the Metadata activity
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(96, 78%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESH%3C/text%3E%3C/svg%3E)
Salomon Henriquez
106
Reputation points Microsoft Employee
Hello!
The run time of the Metadata activity has started to take much longer than before. The activity checks to see if a folder exists or not. This activity has runs of 10 mins, so the pipelines are delayed. Normal behavior for this activity is to take less than a minute. Any idea on what to check or what the reason for the problem could be?
Thanks in advance.
{count} votes
-
AnnuKumari-MSFT 32,161 Reputation points Microsoft Employee2022-11-08T10:49:01.513+00:00 Hi @Salomon Henriquez ,
Thankyou for using Microsoft Q&A platform and thanks for posting your question here.
I understand Get metadata activity is taking long time for completion in your case which is not an expected behaviour. Kindly help me with below details:- Which IR are you using? Is it Auto resolve Integration runtime or self hosted IR?
- Since when are you seeing this behavior of the activity? Were other pipelines running in parallel at the time when this slowness was observed?
-
Salomon Henriquez 106 Reputation points Microsoft Employee
2022-11-08T17:04:00.757+00:00 Hello @ AnnuKumari-MSFT ,
Thank you for your reply.
I'm using Auto resolve IR, the problem started on October 24th and other pipelines are running with no issues in parallel at the time the issue happens.
As a matter of fact, the normal execution time should be 1 minute but, in some cases, this is taking up to 20 minutes and even more.
I noticed this is happening when running the "exists" parameter to determine if a folder exists or not. -
AnnuKumari-MSFT 32,161 Reputation points Microsoft Employee
2022-11-22T07:09:18.397+00:00 Hi @Salomon Henriquez ,
Apologies for delay in response. I tried to replicate your issue, however, I didn't face any delay or slowness. For me , the activity execution got completed within 10 seconds. I Would recommend you to please file a support ticket for deeper investigation and in case if you don't have a support plan, do let us know here so that we can check on other options to unblock you.
Thanks and Regards,
AnnuKumari-MSFT
Sign in to comment