Storage event trigger is running twice when a single blob is created

Question

Hello,

Hope you are fine and you can help me!

I had to modify an Azure function, so i deployed a copy of it from my VSCode. I didn´t realize that I was overwriting all the App Files of the original Azure Function. For example, the host.json file was modified.

The output of my function creates a blob in a container. My issue strats here. I have a trigger (Storage event) in Azure Data Factory that executes a pipeline when the blob is created. However, after I deployed the function and lost the App Files, the trigger is detecting two events.

I suspect this is due to a temporary file being created in the container while the blob finishes. So the blob runs the trigger (eventhough it has no bytes) and then runs again.

Best Regards,

Marco.

Answer 1

https://stackoverflow.com/questions/64970087/azure-event-trigger-fires-multiple-times-same-file

The issue you're encountering, where the Azure Data Factory Storage Event trigger is firing twice for a single blob creation, might be due to the way the Azure Event Grid (AEG) functions. Once the event grid triggers an Azure Function for an event, it expects a response from the Azure Function within 2 minutes. If there is no response, the Event Grid will retry, leading to the perception of a second trigger event. You can mitigate this by changing the default value of retry for the Event Grid from 30 to 1. This should ensure that even for larger files that process for more than 2 minutes, a second duplicate trigger will not occur

Another recommendation is to use the Push-Pull pattern, where the AEG delivers an event to the queue storage and then based on the needs can be processed by Azure Queue Trigger function in a concurrent manner. This pattern provides better control over event processing and avoids the potential for duplicate triggers

https://stackoverflow.com/questions/47398333/create-a-temporary-blob-on-azure-blob-storage

As for the possible cause of the issue being related to a temporary file being created in the container while the blob finishes, Azure Blob Storage operations are atomic. Blocks are uploaded in an uncommitted state, and the final step commits them by providing a list of their IDs. If the final step doesn't happen, you won't see the blob at all unless you explicitly ask for "uncommitted blobs". Thus, it's unlikely that a temporary blob is the cause of the double triggering you're experiencing