Storage event trigger is running twice when a single blob is created

Question

Storage event trigger is running twice when a single blob is created

Marco Manfredi 0

Hello,

Hope you are fine and you can help me!

I had to modify an Azure function, so i deployed a copy of it from my VSCode. I didn´t realize that I was overwriting all the App Files of the original Azure Function. For example, the host.json file was modified.

The output of my function creates a blob in a container. My issue strats here. I have a trigger (Storage event) in Azure Data Factory that executes a pipeline when the blob is created. However, after I deployed the function and lost the App Files, the trigger is detecting two events.

I suspect this is due to a temporary file being created in the container while the blob finishes. So the blob runs the trigger (eventhough it has no bytes) and then runs again.

Best Regards,

Marco.

MuthuKumaranMurugaachari-MSFT 22,451 Reputation points Moderator

2023-06-07T15:51:12.6266667+00:00

Marco Manfredi Thanks for posting your question in Microsoft Q&A. Based on your description, you have modified the Functions code and after the deployment, Storage Event Trigger in Data factory is executing twice. I assume you configured ignoreEmptyBlobs=true as described in doc https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-event-trigger?tabs=data-factory#json-schema to ignore empty blobs. Also, you can filter events for CopyBlob, PutBlob, PutBlockList or FlushWithClose REST API calls in Event Grid so that Microsoft.Storage.BlobCreated event is triggered only when it is completely committed. If not, try configuring the settings and validate.

Do you see time difference between both events is greater than 30 seconds? Event Grid waits for 30 seconds before performing the retry as per Retry schedule.

If you can share the code snippet of Functions (refer Azure Blob storage output binding) and how Event Grid, /ADF is configured, it will be helpful to understand better.
Marco Manfredi 0 Reputation points

2023-06-07T16:05:39.9666667+00:00

hi MuthuKumaranMurugaachari,
Thanks for your time!
1- You Assume Correctly, I configured ignoreEmptyBlobs=true.
2- The time difference is less than 30 second, for example in the last issue it was just 9 seconds.

What about hte Event Grid? Should a modify something?
Below you can find the code in which the function creates the blob.

Regards,
Marco.
MuthuKumaranMurugaachari-MSFT 22,451 Reputation points Moderator

2023-06-07T16:27:33.0833333+00:00

Marco Manfredi Do you have Diagnostic settings enabled on Event Grid https://learn.microsoft.com/en-us/azure/event-grid/enable-diagnostic-logs-topic? It will help in identifying the source and it's event types. I will reach out via email to get instance details and review the backend logs.
Marco Manfredi 0 Reputation points

2023-06-07T17:08:33.7166667+00:00

No, i dont have anything on event grid.

2 answers

Your answer

MuthuKumaranMurugaachari-MSFT 22,451 Reputation points Moderator

2023-06-07T15:51:12.6266667+00:00

Marco Manfredi Thanks for posting your question in Microsoft Q&A. Based on your description, you have modified the Functions code and after the deployment, Storage Event Trigger in Data factory is executing twice. I assume you configured ignoreEmptyBlobs=true as described in doc https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-event-trigger?tabs=data-factory#json-schema to ignore empty blobs. Also, you can filter events for CopyBlob, PutBlob, PutBlockList or FlushWithClose REST API calls in Event Grid so that Microsoft.Storage.BlobCreated event is triggered only when it is completely committed. If not, try configuring the settings and validate.

Do you see time difference between both events is greater than 30 seconds? Event Grid waits for 30 seconds before performing the retry as per Retry schedule.

If you can share the code snippet of Functions (refer Azure Blob storage output binding) and how Event Grid, /ADF is configured, it will be helpful to understand better.
Marco Manfredi 0 Reputation points

2023-06-07T16:05:39.9666667+00:00

hi MuthuKumaranMurugaachari,
Thanks for your time!
1- You Assume Correctly, I configured ignoreEmptyBlobs=true.
2- The time difference is less than 30 second, for example in the last issue it was just 9 seconds.

What about hte Event Grid? Should a modify something?
Below you can find the code in which the function creates the blob.

Regards,
Marco.
MuthuKumaranMurugaachari-MSFT 22,451 Reputation points Moderator

2023-06-07T16:27:33.0833333+00:00

Marco Manfredi Do you have Diagnostic settings enabled on Event Grid https://learn.microsoft.com/en-us/azure/event-grid/enable-diagnostic-logs-topic? It will help in identifying the source and it's event types. I will reach out via email to get instance details and review the backend logs.
Marco Manfredi 0 Reputation points

2023-06-07T17:08:33.7166667+00:00

No, i dont have anything on event grid.

Answer 1

Eyuwankg S Swankg 0

Hi, were you able to solve this issue? Currently facing this problem in my pipelines and trying out different solutions.

Let me know how to fixed it?

Marco Manfredi 0

Hi Eyuwankg,

I solved this issue updating the host.json file. The extensionBundle should have the correct version. In my case, the input "[2.*, 3.0.0)", solved my issue.

Regards,

Marco.

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[2.*, 3.0.0)"
  },
  "functionTimeout": "05:00:00"
}

Answer 2

https://stackoverflow.com/questions/64970087/azure-event-trigger-fires-multiple-times-same-file

The issue you're encountering, where the Azure Data Factory Storage Event trigger is firing twice for a single blob creation, might be due to the way the Azure Event Grid (AEG) functions. Once the event grid triggers an Azure Function for an event, it expects a response from the Azure Function within 2 minutes. If there is no response, the Event Grid will retry, leading to the perception of a second trigger event. You can mitigate this by changing the default value of retry for the Event Grid from 30 to 1. This should ensure that even for larger files that process for more than 2 minutes, a second duplicate trigger will not occur

Another recommendation is to use the Push-Pull pattern, where the AEG delivers an event to the queue storage and then based on the needs can be processed by Azure Queue Trigger function in a concurrent manner. This pattern provides better control over event processing and avoids the potential for duplicate triggers

https://stackoverflow.com/questions/47398333/create-a-temporary-blob-on-azure-blob-storage

As for the possible cause of the issue being related to a temporary file being created in the container while the blob finishes, Azure Blob Storage operations are atomic. Blocks are uploaded in an uncommitted state, and the final step commits them by providing a list of their IDs. If the final step doesn't happen, you won't see the blob at all unless you explicitly ask for "uncommitted blobs". Thus, it's unlikely that a temporary blob is the cause of the double triggering you're experiencing

Marco Manfredi 0 Reputation points

2023-06-07T14:57:14.3966667+00:00

It cant by the AEG because the time between both events is less than a minute.

Share via

Storage event trigger is running twice when a single blob is created

2 answers

Your answer