Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
6,075 questions
Delayed Processing of Initial Queue Messages in WebJob: Seeking Solution for Idle Time in Storage Queue
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 54%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Ajit Soutadekar
1
Reputation point
I have an App Service: AppService1
Service Plan: AppService-Dev-SP (B1: 1)
Configuration->General Settings -> Always On is selected.
I have a Webjob in this service: AppSubscribers
It picks messages from Queue Storage: AppStoragedev
ISSUE:
The 1st message which webjob picks from queue storage- takes 10-20 secs to process.
Subsequent messages are being processed as expected.
Maybe the Storage queue is going in Idle mode.
Need help
Need Solution:
Can you please suggest a suitable solution to handle this idle time in the storage queue?
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 59%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
SnehaAgrawal-MSFT 16,026 Reputation points Microsoft Employee2023-11-01T09:06:45.4866667+00:00 Ajit Soutadekar Thanks for reaching here!
Based on the information provided, it seems like the first message picked up by the WebJob takes longer to process than the subsequent messages.
This could be due to the Storage Queue going into idle mode.
One solution to handle this idle time in the Storage Queue is to increase the visibility timeout of the messages.
To explain-
Azure Storage Queue does not provide Ordering guarantees. However, Storage bus queue is able to provide First In First Out (FIFO) capabilities by using message sessions.
Please check this document that compares both the queues: https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-azure-and-service-bus-queues-compared-contrasted
Messages in Storage Queues are typically first-in-first-out, but sometimes they can be out of order.
For example, when the visibility-timeout duration of a message expires because a client application crashed while processing a message. When the visibility timeout expires, the message becomes visible again on the queue for another worker to dequeue it. At that point, the newly visible message might be placed in the queue to be dequeued again.
By default, the visibility timeout is set to 30 seconds, which means that if a message is picked up by a WebJob and not deleted within 30 seconds, it becomes visible to other clients.
You can increase the visibility timeout to a higher value, such as 2-3 minutes, to give the WebJob enough time to process the message before it becomes visible to other clients. This can be done by setting the
visibilityTimeoutproperty when calling thereceiveMessagemethod in your WebJob code.Another solution is to use the
updateMessagemethod to periodically update the message's visibility timeout while it is being processed by the WebJob. This will prevent the message from becoming visible to other clients while it is still being processed.See- Below links might be helpful
- https://learn.microsoft.com/en-us/azure/storage/queues/storage-quickstart-queues-dotnet?tabs=passwordless%2Croles-azure-portal%2Cenvironment-variable-windows%2Csign-in-azure-cli#receive-messages-from-a-queue
- https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.webjobs.host.queuesoptions.visibilitytimeout?view=azure-dotnet
- https://learn.microsoft.com/en-us/rest/api/storageservices/put-message I hope this helps! Let me know if you have any further questions.
-
Ajit Soutadekar 1 Reputation point
2023-11-01T09:20:55.59+00:00 @SnehaAgrawal-MSFT Thank you for your response.
As per your suggestion: I need to increase the value of QueuesOptions.VisibilityTimeout Property example 2-3 minutes.
I understood the logic of increasing this timeout: Webjob will get more time to process the message.
But not able to understand how this is related to the Storage queue going idle mode. Can you please explain why the storage queue is going into idle mode?
This is happening frequently:
- The queue goes into idle mode.
- The first message takes time to process.
- Subsequent messages are processed in time.
- After some idle time: the first message after this idle time again takes more time to process. Subsequent messages are processed in time. And So on.
-
SnehaAgrawal-MSFT 16,026 Reputation points Microsoft Employee
2023-11-02T12:45:14.21+00:00 Ajit Soutadekar Could you please confirm if Always On is ON?
Also, if you are using a Connection String instead of an App Setting from the storage connection, if not could you try switching to an App Setting?
See- #1240 (comment)
-
Ajit Soutadekar 1 Reputation point
2023-11-02T13:06:58.1266667+00:00 - @SnehaAgrawal-MSFT As mentioned in my main question: Always ON is ON
- Connection String instead of an App Setting: The comment that you are referring to is for Azure Function App. The connection string setting is w.r.t. function app. I am not using function apps.
- Azure Storage -> Access Keys -> Key1 -> connection string
- We get this connection string and add it to our appsettings.json file of the webjob.
- Please let me know if you need any more information.
Thank you for your guidance.
-
SnehaAgrawal-MSFT 16,026 Reputation points Microsoft Employee
2023-11-07T07:31:27.9533333+00:00 @Ajit Soutadekar Apologies you are facing issue here! Could you please reach out to us by sending an email with subject line “Attn:Sneha |thread title” to AzCommunity[at]microsoft[dot]com referencing this thread, for deeper investigation on the issue.
-
Ajit Soutadekar 1 Reputation point
2023-11-07T07:35:40.3933333+00:00 @SnehaAgrawal-MSFT : Thank you for your response. Will share full details in the mail.
-
SnehaAgrawal-MSFT 16,026 Reputation points Microsoft Employee
2023-11-09T07:03:00.9333333+00:00 @Ajit Soutadekar Could you please send us email on with subject line “Attn:Sneha |thread title” to AzCommunity[at]microsoft[dot]com referencing this thread, for deeper investigation on the issue.
Sign in to comment