Azure Blob Storage trigger Function not firing when upload a file about 2GB but run with 1GB

Question

I have an Azure function that runs with a blob trigger. When I upload a file of about 1 GB it's ok and runs. But when I upload a file of about 2GB the trigger doesn't fire.
Is there any limitation or configuration?

Answer 1

@Mohsen Akhavan ,

Thanks for reaching out to Q&A.

In general, there is no size limit on functions, however we need to understand how the trigger works. By defalt the trigger passes in a "string" or a "byte[]" of data, which means all the data is loaded in memory and then passed to the function. Usually customers hit an "out of memory" exception when the content gets to big. To get around this in .NET at least, you can pass in the type of Stream so that the data isn't fully loaded in but streamed in (and ideally out) of the function and can process massive files. We have tested with a 4GB file without any issues, even though I only had 1.5 GB of memory (consumption plan).

However I am not sure if Python / JavaScript / Java / PowerShell support Stream. In case not what I'd recommend is:
• Trigger using the Event Grid trigger on a storage blob create event. This will still trigger on a blob, but won't pass in any content. Just some data about which blob was created
• Use the Python SDKs directly in the funciton code to stream in the blob data. Be sure they stream it though - can't load 10GB in memory

Additionally there are sync and async versions of the Blob Storage library. Use the asynchronous aio version for best performance. Use the streaming methods such as chunks() to stream without loading everything into memory at once.

I hope this helps!

Please 'Accept as answer' and ‘Upvote’ if it helped so that it can help others in the community looking for help on similar topics.