Azure Function Azure Blob Storage trigger type for extremely large blob

Question

Hi team,

I've been experimenting with Azure Blob Storage triggers using v2 programming model. The blobs I'm uploading to the blob storage are about 100 GB. I've tested 4 possibilities to get an event about the blob change:

• InputStream - resulted in error System.OutOfMemoryException
• BlobClient and ContainerClient - triggered successfully and events came very fast
• StorageStreamDownloader - triggered with a 40 min delay

I would definitely choose BlobClient as trigger type just to get the event and then do the processing on my own.
My question is - how to achieve this behaviour for v1 programming model? In the docs I can see only InputStream is supported. What's your recommendation if the blob won't fit into memory?

Thanks a lot,
Maria

Answer 1

Hi Maria Dąbrowiecka,

It looks like your approach using the BlobClient as a trigger in the v2 programming model is absolutely correct. You're right in choosing BlobClient, as it allows you to get the event and then proceed with your own processing without loading the entire blob into memory, which is crucial for handling large files like the 100GB blobs you're working with.

Achieving the same behavior in the v1 programming model, however, can be challenging. As you've noted, the v1 model primarily supports InputStream as the trigger, which requires loading the blob content into memory. For blobs that are too large to fit into memory, this can result in memory-related exceptions, such as the System.OutOfMemoryException you've encountered.

Unfortunately, the v1 programming model does not natively support the same flexible approach to triggering events based on BlobClient like the v2 model does. There isn't a simple built-in solution for handling large blobs without reading them into memory.

For more details on what’s supported in the v1 programming model, I recommend referring to the following docs:

• Azure Functions Triggers and Bindings (v1)
• BlobTrigger (v1)

Hope this helps! Let me know if you have any further questions or need assistance.