Blob trigger azure function using same queue name for all available blob triggers?

Question

Blob trigger azure function using same queue name for all available blob triggers?

Keerthana, D 40

Hi Team,

In our current Azure Function setup, we are utilizing multiple Blob triggers—specifically, 3 to 4 triggers—each pointing to distinct storage containers. However, during execution, all these Blob triggers internally reference the same trigger configuration, namely "azure-webjobs-blobtrigger-cdmsgenai-dev-api" in our environment. If we tries to change the queue name with QueueClient internally it again calls the default queue only.

This overlap is causing a bottleneck, preventing simultaneous processing of the Blob trigger functionalities. I recommend we explore a solution that allows for parallel execution without conflict, ensuring each Blob trigger operates independently.

Looking forward to your thoughts.

Keerthana, D 40 Reputation points

2024-11-02T10:43:14.0233333+00:00

@Deepanshukatara-6769 Dynamic concurrency has been enabled in host.json but could you suggest some methods for verifying concurrency while testing locally.
Khadeer Ali 5,990 Reputation points Microsoft External Staff Moderator

2024-11-05T00:56:45.5433333+00:00

HI @Deepanshukatara-6769,

Could you please take a look at Keerthana’s question? Your insights would be greatly appreciated!
Deepanshukatara-6769 16,405 Reputation points Moderator

2024-11-05T06:15:35.0166667+00:00
Hello Keerthana ,

To test how your Azure Function handles concurrency by generating multiple requests simultaneously, you can create a script that sends multiple POST requests to your function endpoint. Below is an example using Python and the requests library to achieve this:

import requests import threading # Define the function endpoint URL function_url = "http://localhost:7071/api/YourFunctionName" # Define the payload to send in the POST request payload = { "key1": "value1", "key2": "value2" } # Define the number of concurrent requests num_requests = 10 # Function to send a POST request def send_post_request(): response = requests.post(function_url, json=payload) print(f"Response Status Code: {response.status_code}, Response Body: {response.text}") # Create a list to hold the threads threads = [] # Create and start threads to send concurrent requests for _ in range(num_requests): thread = threading.Thread(target=send_post_request) threads.append(thread) thread.start() # Wait for all threads to complete for thread in threads: thread.join() print("All requests completed.")

The function_url variable holds the URL of your Azure Function endpoint.

The payload variable contains the data to be sent in the POST request.

The num_requests variable defines the number of concurrent requests you want to send.

The send_post_request function sends a POST request to the function endpoint and prints the response status code and body.

A list of threads is created, and each thread is started to send a POST request concurrently.

The script waits for all threads to complete before printing "All requests completed."

By running this script, you can simulate multiple concurrent requests to your Azure Function and observe how it handles concurrency.
Khadeer Ali 5,990 Reputation points Microsoft External Staff Moderator

2024-11-06T01:25:12.06+00:00

Hi Keerthana, D,
Just checking in to see if the below answer provided by Deepanshukatara-6769 helped.

If this answers your query, do click Comment and Upvote it for was this answer helpful. And, if you have any further query do let us know.

Accepted answer

0 additional answers

Your answer

Keerthana, D 40 Reputation points

2024-11-02T10:43:14.0233333+00:00

@Deepanshukatara-6769 Dynamic concurrency has been enabled in host.json but could you suggest some methods for verifying concurrency while testing locally.
Khadeer Ali 5,990 Reputation points Microsoft External Staff Moderator

2024-11-05T00:56:45.5433333+00:00

HI @Deepanshukatara-6769,

Could you please take a look at Keerthana’s question? Your insights would be greatly appreciated!
Deepanshukatara-6769 16,405 Reputation points Moderator

2024-11-05T06:15:35.0166667+00:00

Hello Keerthana ,

To test how your Azure Function handles concurrency by generating multiple requests simultaneously, you can create a script that sends multiple POST requests to your function endpoint. Below is an example using Python and the requests library to achieve this:

import requests import threading # Define the function endpoint URL function_url = "http://localhost:7071/api/YourFunctionName" # Define the payload to send in the POST request payload = { "key1": "value1", "key2": "value2" } # Define the number of concurrent requests num_requests = 10 # Function to send a POST request def send_post_request(): response = requests.post(function_url, json=payload) print(f"Response Status Code: {response.status_code}, Response Body: {response.text}") # Create a list to hold the threads threads = [] # Create and start threads to send concurrent requests for _ in range(num_requests): thread = threading.Thread(target=send_post_request) threads.append(thread) thread.start() # Wait for all threads to complete for thread in threads: thread.join() print("All requests completed.")

The function_url variable holds the URL of your Azure Function endpoint.

The payload variable contains the data to be sent in the POST request.

The num_requests variable defines the number of concurrent requests you want to send.

The send_post_request function sends a POST request to the function endpoint and prints the response status code and body.

A list of threads is created, and each thread is started to send a POST request concurrently.

The script waits for all threads to complete before printing "All requests completed."

By running this script, you can simulate multiple concurrent requests to your Azure Function and observe how it handles concurrency.
Khadeer Ali 5,990 Reputation points Microsoft External Staff Moderator

2024-11-06T01:25:12.06+00:00

Hi Keerthana, D,
Just checking in to see if the below answer provided by Deepanshukatara-6769 helped.

If this answers your query, do click Comment and Upvote it for was this answer helpful. And, if you have any further query do let us know.

Answer 1

Hello , Welcome to MS Q&A

To configure multiple Blob/Queue triggers in Azure Functions and avoid execution bottlenecks, you can utilize dynamic concurrency. This feature allows the Azure Functions runtime to manage the number of concurrent executions dynamically based on the workload. By enabling dynamic concurrency, you can ensure that your functions can scale out to handle multiple blob events simultaneously, reducing the likelihood of bottlenecks.

Additionally, consider using the Blob storage trigger that consumes events from an event subscription, which can provide lower latency and better performance compared to polling methods. This approach can help in efficiently processing blobs as they are added or updated.

References:

Please let me know if any further questions

Please accept answer

Thanks
Deepanshu

Share via

Blob trigger azure function using same queue name for all available blob triggers?

0 additional answers

Your answer