Question on Event Hub

Question

Question on Event Hub

Malik, Deepankar 25

Hi, I am using Azure Function App with Isolated App service plan. I have created Event Hub Trigger Function in that function app.

I want to know how many parallel executions my Event Hub trigger can achieve. If this depends upon certain factors, then I would like to know about them.

Event Hub is on dedicated tier and has two partition count and one consumer group.

Thanks, Deepankar

Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-03-19T19:26:06.09+00:00

Hello Malik, Deepankar,

A similar thread has been discussed here.

it seems like there isn’t a way to specify a maximum concurrency for Event Hubs triggered functions, but you can control batch size and fetching options. The maximum number of concurrent invocations may also vary depending on your workload and resource utilization.

Considering your Event Hub is on a dedicated tier with two partition counts and one consumer group, you should also consider the throughput units and the number of partitions, as they play a significant role in the scalability and performance of your Event Hub. Each partition can only be read by a single consumer at a time, so having more partitions can allow for greater parallelism and throughput.

To optimize for parallel executions, ensure that your function app is scaled appropriately, monitor the performance metrics, and adjust the configurations as necessary. You can also consider using Azure Functions Premium plan, which provides higher concurrency and throughput limits compared to the Consumption plan.

I hope this helps.
Malik, Deepankar 25 Reputation points

2024-03-20T19:31:34.5966667+00:00

@Bhargava-MSFT Thanks for your response.

You mentioned that maximum number of concurrent invocations may also vary depending on your workload and resource utilization, I need to process events at the rate of 500 per second. I just want to know if it makes any difference by scaling up considering the resource utilization is less than 50%(CPU and memory) or I simply play with maxBatchSize and other settings in host.json to fulfil my requirement?

Do we have any correlation between number of cores and event hub trigger parallel executions?
Malik, Deepankar 25 Reputation points

2024-03-21T21:20:54.9266667+00:00

@Bhargava-MSFT Can you please clarify my earlier asked qestions
?
Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-03-22T21:49:02.85+00:00

If your CPU and memory utilization is less than 50%, scaling up might not significantly improve throughput, as there are ample resources available for processing. Instead, I recommend focusing on optimizing specific configuration settings in your host.json file, such as adjusting parameters like maxBatchSize and prefetchCount. These adjustments allow you to fine-tune how your function app interacts with the Event Hub, potentially improving parallel execution and throughput.

Regarding the correlation between the number of cores and Event Hub trigger parallel executions, it's important to note that while having more cores can increase the potential for parallel executions, it's not the sole determining factor. Factors such as the number of partitions, throughput units, and overall resource utilization also play significant roles in scalability and performance.

Additionally, ensure that your function code is optimized for performance. Avoiding long-running or blocking operations within your function code can help maximize concurrency and throughput.

I hope this helps.
Malik, Deepankar 25 Reputation points

2024-03-26T19:47:32.8833333+00:00

@Bhargava-MSFT Do we have any metric that can show number of parallel executions per instance?