Azure data factory : storage event trigger issue. the configured storage event triggering twice.

Question

Hi Everyone,
Facing issues in production environment with storage event trigger type. we have configured a trigger to monitor a
container: test
blobpath begin with
foldername/Input/file1

blob path ends with
.txt

sample file name: file1.txt

as soon as file copying by a FTP process to the above test container and foldername/Input/ folder, our process is firing automatically. Few days it's working fine.. but some times it is failing. Not sure whether it's a FTP process copying same file twice in same location with seconds difference or not. Unable debug that issue.

for time being we have added concurrency as 1 and first process deleting the file from input folder after pipeline execution, second instance of pipeline which one triggering is in waiting queue and going to IN progress after first process completion. As file not available in blob, second process failing. so this way we are avoiding the issue.

but need to find out whether FTP process copying twice or not. anywhere i can see blob copy logs in the azure monitor section?

Answer 1

Hello @Pavan Kumar Chakilam ,

Thanks for the question and using MS Q&A platform.

Seems like your while FTP is trying to write the file ADF is considering that as a new file before it is being completed written and hence you might be seeing this behavior.

Could you please try to set 'Ignore empty blobs to true' in you event trigger configuration and see if that helps to overcome the issue.

In order to monitor which file has initiated the trigger to fire, you can set the Trigger parameters for folder path and file name as below and whenever the trigger is being fired it will automatically initiate a pipeline run which is associated with it. If that is the case, then you should be able to see the file name from the Monitor section by going to that particular trigger and from their navigating to the associated pipeline run and checking the parameters values as shown below.

Hope this info helps to overcome the issue. Let us know how it goes.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how