How to prevent new blob trigger function app triggering on existing blobs

Question

Hi,

I am developing a new function app to trigger when a new blob is added to a storage container that already has existing blobs within it.

When testing locally, the app is triggering on all existing blobs and I don't want the app to trigger on and process any existing blobs due to the nature of the new app.

(The intended outcome for each trigger is a DB update, and I can't find a way to ensure the blobs will be processed in order, such that the DB will be up to date and correct after processing all existing blobs - e.g. the blob from 2 days ago could be processed after the most up-to-date blob, thus meaning the DB would contain data from 2 days ago instead of the most up-to-date data).

Is it possible to prevent processing of existing blobs? Or alternatively, is it possible to ensure blobs are processed in a particular order?

I'm also curious as to whether all existing blobs would be re-processed every time an update is made to the app or when it is restarted? This would also be undesirable as it would involve unnecessary processing.

Thanks,

Luke

Answer 1

@Luke Osborn Thanks for reaching out.

If you are using the storage blob trigger, then your function app should be triggered only for when a new or updated blob is detected once you have configured your function app. It should not be triggered for the existing blobs if there is no update to existing blobs. Please validate if this is not the case for your observation.

Is it possible to ensure blobs are processed in a particular order? No there is no configuration to process the blobs in order.

I'm also curious as to whether all existing blobs would be re-processed every time an update is made to the app or when it is restarted?

No, it should not trigger for existing blobs until and unless there is update blob on the existing blobs.

If you are only looking for BlobCreated event then you should leverage the event grid for storage event with azure function

Blob (Microsoft.Storage.BlobCreated) event --> Event Grid --> Function

For more details you can refer to this.

If you want to ensure that the Microsoft.Storage.BlobCreated event is triggered only when a Block Blob is completely committed, filter the event for the CopyBlob, PutBlob, PutBlockList or FlushWithClose REST API calls. These API calls trigger the Microsoft.Storage.BlobCreated event only after data is fully committed to a Block Blob. To learn how to create a filter, see Filter events for Event Grid.