Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Executing a databricks notebook in queue for more than an hour. The cluster is up and running.
Data factory lookup activity timeout
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 52%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAV%3C/text%3E%3C/svg%3E)
Ashish Venugopal
6
Reputation points
Getting a strange error since last few days. I have a lookup activity which fetch a parameter from azure SQL which is currently in serverless tier.
The timeout duration is given as 5 minutes but it still it fails,even retry of 3 doesn't work. This happens when the data factory is triggered on schedule
But if i run the same pipeline in debug mode everything works just fine.
We are having issues in live region as well where the SQL server is on a higher tier.
Can someone help?
Thanks in advance.
Azure Data Factory
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 70%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
MartinJaffer-MSFT 26,161 Reputation points2022-06-23T19:03:16.78+00:00 Hello @Ashish Venugopal ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .Thanks
Martin
Sign in to comment
4 answers
Sort by: Most helpful
-
Ashish Venugopal 6 Reputation points
2022-06-29T08:41:49.937+00:00 @MartinJaffer-MSFT why does most of the activities get in Queue for very long time ?
-
Ashish Venugopal 6 Reputation points
2022-06-29T08:27:08.507+00:00 Hi @MartinJaffer-MSFT ,
Thanks for the reply. The same published version is ran for both trigger and debug. This is happening quite frequently now a days. The lookup activity has a timeout of 5 mins and the activity is in queue for 5 minutes and it fails with timeout. We have same data factory running in multiple subscriptions and the same error happens in each suscription. -
MartinJaffer-MSFT 26,161 Reputation points
2022-06-20T20:29:20.133+00:00 Hello @Ashish Venugopal ,
Thanks for the question and using MS Q&A platform.I understand you are using Data Factory Lookup activity targeting Azure SQL. When you do a debug run, everything is fine, but when you do a trigger run, it times out? Can you tell me how long it takes in the debug run?
The first thing that comes to mind, is a possible difference between the version you are running debug on, and the published version. Debug runs, use whatever you see in the browser at the moment. Trigger runs use whatever was published last. Try publishing and see whether that evens things out.
We are having issues in live region as well where the SQL server is on a higher tier.
does this mean you are having two separate instances of the problem? This changes things. I do know that debug and trigger run on different computes.
Please do let me if you have any queries.
Thanks
Martin------------------------------------------
- Please don't forget to click on
or upvote 
button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how - Want a reminder to come back and check responses? Here is how to subscribe to a notification
- Please don't forget to click on