Share via

Lookup Activity not returning data

Abdul Rahman T 20 Reputation points
2024-09-13T05:53:11.4633333+00:00

When I query Snowflake database using lookup activity, I am not getting the result back. So, the pipeline is failing. But I run again immediately I am getting the data in lookup activity. I used the option first row only. I am getting this issue after updating to latest snowflake connector.

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

0 comments

Answer accepted by question author

Amira Bedhiafi 42,941 Reputation points MVP Volunteer Moderator
2024-09-13T07:52:49.08+00:00

The first run might be happening during a time when the data in Snowflake is still being updated or not fully committed, which causes the Lookup activity to fail. The second run works because the data becomes available.

I recommend that you implement a retry mechanism in your pipeline using an "Until" activity or a retry policy for the Lookup activity, allowing it to reattempt fetching the data until it succeeds.

You could also try downgrading to the previous version temporarily to see if it resolves the issue.

Was this answer helpful?

0 comments

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.