Storage event trigger is running twice when a single blob is created

Osamwonyi Alekwe 40 Reputation points
2024-02-16T08:11:41.2966667+00:00

Hi All,   I created a storage event trigger on a pipeline, but noticed that the trigger runs twice when a single file lands in the container , when two file lands the pipeline is triggered 3 times, i expect the pipeline to run once when a single file lands in the storage account. has any one encountered this scenario and how was it resolved

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,348 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,599 questions
{count} votes

Accepted answer
  1. Amrinder Singh 2,195 Reputation points Microsoft Employee
    2024-02-16T09:38:08.77+00:00

    Hello @Osamwonyi Alekwe Welcome to Microsoft Q&A Forum, thank you for posting your query here!

    We would suggest you enable storage diagnostics logging which shall help in reviewing how operations are being performed. This shall help with an isolation further. https://learn.microsoft.com/en-us/azure/storage/blobs/monitor-blob-storage?tabs=azure-portal

    Once logging has been enabled, we need to check on the APIs that are execute underneath.

    If you are using Blob API, such as PutBlob, you can review if you are seeing more than 1 call when the upload is performed. If yes, then this is due to application making 2 requests as a result 2 events are being generated in return.

    If you are using DFS endpoint API such as Create/Append/Flush, then there are 2 events that will get generated one for CreateFile and another one for Flush operation. If that's the case, you can filter only to FlushFile operation as discussed in the below link: https://learn.microsoft.com/en-us/azure/event-grid/event-schema-blob-storage?toc=%2Fazure%2Fstorage%2Fblobs%2Ftoc.json&tabs=event-grid-event-schema

    Please let us know if you have any further queries. I’m happy to assist you further.

    ---Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.


1 additional answer

Sort by: Most helpful
  1. Amrinder Singh 2,195 Reputation points Microsoft Employee
    2024-02-20T10:16:28.6166667+00:00

    Hello @Osamwonyi Alekwe - This is just a follow up to check if got opportunity to review this further.