Standard plan EventHub event size limit is not 1mb when posted from Azure Function

Question

Recently I've been smacked with exception saying:

Exception while executing function: Event is too large. Event is approximately 667777b and max size is 245760b

it was thrown from IAsyncAcumulator which was EventHub binding of Auzre Function. EventHub is in standard plan and according to docs allowed event size should be up to 1MB but exception claims it is 240kB, which resembles 256kB of basic plan. I've contacted Azure Support on Twitter with it as apparently I am missing something. After brief moment of digging they responded with statement:

The max size of your pipeline is 245760b and you are at 667777b (...)

I was not aware of any pipeline involved in it and when I asked about resources describing it to fill holes in my knowledge, I got:

We aren't able to find any tutorials/set up for processing large volumes of transactions for Azure functions. You may need to do some backend configuration on this for the message to be processed via batches in order to lower the volume. (...)

and asked to post my question here to ask for further insights.

So here I am, I would like to know what I am missing to be able to utilize full might and splendor of Event Hub in standard plan ;) Any links to resources would be invaluable. Oh and thanks in advance!

Answer 1

Hi @Adrian Gawor

Apology for the issue that you are facing.

I was able to repro the issue at my end and have looked into the source code and found that the error was due to the validation at line no 142 and the error was thrown by the line no 145. The value is hardcoded as 240 KB. I have created the PR either to fix it or dynamically fetch it according to the event hub SKU.

Update:
I have got an update from my team that this is by design and they are focusing on deciding when to implement the functionality as discussed in this item.

Please 'Accept as answer' and ‘Upvote’ if it helped so that it can help others in the community looking for help on similar topics.

Answer 2

Looks like this was fixed in v4.1.2 on the eventhub binding: https://github.com/Azure/azure-functions-eventhubs-extension/pull/77