Datafactory Lookup Actitivity with Snowflake Connector - Unstable performance

Question

Hi!

Since a couple of days back our pipelines are getting stucked due to long execution times of Lookup activities connected to Snowflake. The Linked-service is connected via AutoResolveIntegrationRuntime.

When I check the statements progress in my snowflake environment, these are full executed pretty much just after they were called (less than 1 second). Nevertheless, corresponding Data Factory's activity can be stuck in status "In Progress" for over 1 hour.
This behavior is completely random and happens sometimes.
It looks that there is a completely missynced between Snowflake and ADF?

Server: West Europe.
More background: This is new, the pipelines have been running over months without issues. Same behavior is observed among 5 different subscriptions, all of them hosted in West Europe server.

I would appreciate your insights.
Thanks in advanced.
Gilfredo

Answer 1

Hi,

Thank you for posting query in Microsoft Q&A Platform.

I got update from internal team, that West Europe has been little problematic zone for all Azure Services in Jan. There are few instances where one customer got Lookup activity timeout of Azure networking issues with Azure SQL and ADF. In your case, snowflake completed your task, but ACK broken. Hopefully, now if you try you may not see the issue.

Kindly try now and see how it behaves. If it's still broken, then I would encourage you to have a support ticket for deeper root cause investigation. Below is the link for creating support ticket.

https://azure.microsoft.com/en-us/support/create-ticket

Hope this helps.

---

Please consider hitting Accept Answer button. Accepted answers help community as well.