Azure Queue storage trigger for Azure Functions

The queue storage trigger runs a function as messages are added to Azure Queue storage.

Azure Queue storage scaling decisions for the Consumption and Premium plans are done via target-based scaling. For more information, see Target-based scaling.

Important

This article uses tabs to support multiple versions of the Node.js programming model. The v4 model is generally available and is designed to have a more flexible and intuitive experience for JavaScript and TypeScript developers. For more details about how the v4 model works, refer to the Azure Functions Node.js developer guide. To learn more about the differences between v3 and v4, refer to the migration guide.

Azure Functions supports two programming models for Python. The way that you define your bindings depends on your chosen programming model.

The Python v2 programming model lets you define bindings using decorators directly in your Python function code. For more information, see the Python developer guide.

This article supports both programming models.

Example

Use the queue trigger to start a function when a new item is received on a queue. The queue message is provided as input to the function.

A C# function can be created by using one of the following C# modes:

  • Isolated worker model: Compiled C# function that runs in a worker process that's isolated from the runtime. Isolated worker process is required to support C# functions running on LTS and non-LTS versions .NET and the .NET Framework. Extensions for isolated worker process functions use Microsoft.Azure.Functions.Worker.Extensions.* namespaces.
  • In-process model: Compiled C# function that runs in the same process as the Functions runtime. In a variation of this model, Functions can be run using C# scripting, which is supported primarily for C# portal editing. Extensions for in-process functions use Microsoft.Azure.WebJobs.Extensions.* namespaces.

The following example shows a C# function that polls the input-queue queue and writes several messages to an output queue each time a queue item is processed.

[Function(nameof(QueueFunction))]
[QueueOutput("output-queue")]
public string[] Run([QueueTrigger("input-queue")] Album myQueueItem, FunctionContext context)
{
    // Use a string array to return more than one message.
    string[] messages = {
        $"Album name = {myQueueItem.Name}",
        $"Album songs = {myQueueItem.Songs.ToString()}"};

    _logger.LogInformation("{msg1},{msg2}", messages[0], messages[1]);

    // Queue Output messages
    return messages;
}

The following Java example shows a storage queue trigger function, which logs the triggered message placed into queue myqueuename.

@FunctionName("queueprocessor")
public void run(
    @QueueTrigger(name = "msg",
                queueName = "myqueuename",
                connection = "myconnvarname") String message,
    final ExecutionContext context
) {
    context.getLogger().info(message);
}

The following example shows a queue trigger TypeScript function. The function polls the myqueue-items queue and writes a log each time a queue item is processed.

import { app, InvocationContext } from '@azure/functions';

export async function storageQueueTrigger1(queueItem: unknown, context: InvocationContext): Promise<void> {
    context.log('Storage queue function processed work item:', queueItem);
    context.log('expirationTime =', context.triggerMetadata.expirationTime);
    context.log('insertionTime =', context.triggerMetadata.insertionTime);
    context.log('nextVisibleTime =', context.triggerMetadata.nextVisibleTime);
    context.log('id =', context.triggerMetadata.id);
    context.log('popReceipt =', context.triggerMetadata.popReceipt);
    context.log('dequeueCount =', context.triggerMetadata.dequeueCount);
}

app.storageQueue('storageQueueTrigger1', {
    queueName: 'myqueue-items',
    connection: 'MyStorageConnectionAppSetting',
    handler: storageQueueTrigger1,
});

The usage section explains queueItem. The message metadata section explains all of the other variables shown.

The following example shows a queue trigger JavaScript function. The function polls the myqueue-items queue and writes a log each time a queue item is processed.

const { app } = require('@azure/functions');

app.storageQueue('storageQueueTrigger1', {
    queueName: 'myqueue-items',
    connection: 'MyStorageConnectionAppSetting',
    handler: (queueItem, context) => {
        context.log('Storage queue function processed work item:', queueItem);
        context.log('expirationTime =', context.triggerMetadata.expirationTime);
        context.log('insertionTime =', context.triggerMetadata.insertionTime);
        context.log('nextVisibleTime =', context.triggerMetadata.nextVisibleTime);
        context.log('id =', context.triggerMetadata.id);
        context.log('popReceipt =', context.triggerMetadata.popReceipt);
        context.log('dequeueCount =', context.triggerMetadata.dequeueCount);
    },
});

The usage section explains queueItem. The message metadata section explains all of the other variables shown.

The following example demonstrates how to read a queue message passed to a function via a trigger.

A Storage queue trigger is defined in function.json file where type is set to queueTrigger.

{
  "bindings": [
    {
      "name": "QueueItem",
      "type": "queueTrigger",
      "direction": "in",
      "queueName": "messages",
      "connection": "MyStorageConnectionAppSetting"
    }
  ]
}

The code in the Run.ps1 file declares a parameter as $QueueItem, which allows you to read the queue message in your function.

# Input bindings are passed in via param block.
param([string] $QueueItem, $TriggerMetadata)

# Write out the queue message and metadata to the information log.
Write-Host "PowerShell queue trigger function processed work item: $QueueItem"
Write-Host "Queue item expiration time: $($TriggerMetadata.ExpirationTime)"
Write-Host "Queue item insertion time: $($TriggerMetadata.InsertionTime)"
Write-Host "Queue item next visible time: $($TriggerMetadata.NextVisibleTime)"
Write-Host "ID: $($TriggerMetadata.Id)"
Write-Host "Pop receipt: $($TriggerMetadata.PopReceipt)"
Write-Host "Dequeue count: $($TriggerMetadata.DequeueCount)"

The following example demonstrates how to read a queue message passed to a function via a trigger. The example depends on whether you use the v1 or v2 Python programming model.

import logging
import azure.functions as func

app = func.FunctionApp()

@app.function_name(name="QueueFunc")
@app.queue_trigger(arg_name="msg", queue_name="inputqueue",
                   connection="storageAccountConnectionString")  # Queue trigger
@app.write_queue(arg_name="outputQueueItem", queue_name="outqueue",
                 connection="storageAccountConnectionString")  # Queue output binding
def test_function(msg: func.QueueMessage,
                  outputQueueItem: func.Out[str]) -> None:
    logging.info('Python queue trigger function processed a queue item: %s',
                 msg.get_body().decode('utf-8'))
    outputQueueItem.set('hello')

Attributes

Both in-process and isolated worker process C# libraries use the QueueTriggerAttribute to define the function. C# script instead uses a function.json configuration file as described in the C# scripting guide.

In C# class libraries, the attribute's constructor takes the name of the queue to monitor, as shown in the following example:

[Function(nameof(QueueFunction))]
[QueueOutput("output-queue")]
public string[] Run([QueueTrigger("input-queue")] Album myQueueItem, FunctionContext context)

This example also demonstrates setting the connection string setting in the attribute itself.

Annotations

The QueueTrigger annotation gives you access to the queue that triggers the function. The following example makes the queue message available to the function via the message parameter.

package com.function;
import com.microsoft.azure.functions.annotation.*;
import java.util.Queue;
import com.microsoft.azure.functions.*;

public class QueueTriggerDemo {
    @FunctionName("QueueTriggerDemo")
    public void run(
        @QueueTrigger(name = "message", queueName = "messages", connection = "MyStorageConnectionAppSetting") String message,
        final ExecutionContext context
    ) {
        context.getLogger().info("Queue message: " + message);
    }
}
Property Description
name Declares the parameter name in the function signature. When the function is triggered, this parameter's value has the contents of the queue message.
queueName Declares the queue name in the storage account.
connection Points to the storage account connection string.

Decorators

Applies only to the Python v2 programming model.

For Python v2 functions defined using decorators, the following properties on the queue_trigger decorator define the Queue Storage trigger:

Property Description
arg_name Declares the parameter name in the function signature. When the function is triggered, this parameter's value has the contents of the queue message.
queue_name Declares the queue name in the storage account.
connection Points to the storage account connection string.

For Python functions defined by using function.json, see the Configuration section.

Configuration

Applies only to the Python v1 programming model.

The following table explains the properties that you can set on the options object passed to the app.storageQueue() method.

Property Description
queueName The name of the queue to poll.
connection The name of an app setting or setting collection that specifies how to connect to Azure Queues. See Connections.

The following table explains the binding configuration properties that you set in the function.json file and the QueueTrigger attribute.

function.json property Description
type Must be set to queueTrigger. This property is set automatically when you create the trigger in the Azure portal.
direction In the function.json file only. Must be set to in. This property is set automatically when you create the trigger in the Azure portal.
name The name of the variable that contains the queue item payload in the function code.
queueName The name of the queue to poll.
connection The name of an app setting or setting collection that specifies how to connect to Azure Queues. See Connections.

See the Example section for complete examples.

When you're developing locally, add your application settings in the local.settings.json file in the Values collection.

Usage

Note

Functions expect a base64 encoded string. Any adjustments to the encoding type (in order to prepare data as a base64 encoded string) need to be implemented in the calling service.

The usage of the Queue trigger depends on the extension package version, and the C# modality used in your function app, which can be one of the following:

An isolated worker process class library compiled C# function runs in a process isolated from the runtime.

Choose a version to see usage details for the mode and version.

The queue trigger can bind to the following types:

Type Description
string The message content as a string. Use when the message is simple text..
byte[] The bytes of the message.
JSON serializable types When a queue message contains JSON data, Functions tries to deserialize the JSON data into a plain-old CLR object (POCO) type.
QueueMessage1 The message.
BinaryData1 The bytes of the message.

1 To use these types, you need to reference Microsoft.Azure.Functions.Worker.Extensions.Storage.Queues 5.2.0 or later and the common dependencies for SDK type bindings.

The QueueTrigger annotation gives you access to the queue message that triggered the function.

Access the queue item as the first argument to your function. If the payload is JSON, the value is deserialized into an object.

Access the queue message via string parameter that matches the name designated by binding's name parameter in the function.json file.

Access the queue message via the parameter typed as QueueMessage.

Metadata

The queue trigger provides several metadata properties. These properties can be used as part of binding expressions in other bindings or as parameters in your code, for language workers that provide this access to message metadata.

The message metadata properties are members of the CloudQueueMessage class.

The message metadata properties can be accessed from context.triggerMetadata.

The message metadata properties can be accessed from the passed $TriggerMetadata parameter.

Property Type Description
QueueTrigger string Queue payload (if a valid string). If the queue message payload is a string, QueueTrigger has the same value as the variable named by the name property in function.json.
DequeueCount long The number of times this message has been dequeued.
ExpirationTime DateTimeOffset The time that the message expires.
Id string Queue message ID.
InsertionTime DateTimeOffset The time that the message was added to the queue.
NextVisibleTime DateTimeOffset The time that the message will next be visible.
PopReceipt string The message's pop receipt.

The following message metadata properties can be accessed from the passed binding parameter (msg in previous examples).

Property Description
body Queue payload as a string.
dequeue_count The number of times this message has been dequeued.
expiration_time The time that the message expires.
id Queue message ID.
insertion_time The time that the message was added to the queue.
time_next_visible The time that the message will next be visible.
pop_receipt The message's pop receipt.

Connections

The connection property is a reference to environment configuration that specifies how the app should connect to Azure Queues. It may specify:

If the configured value is both an exact match for a single setting and a prefix match for other settings, the exact match is used.

Connection string

To obtain a connection string, follow the steps shown at Manage storage account access keys.

This connection string should be stored in an application setting with a name matching the value specified by the connection property of the binding configuration.

If the app setting name begins with "AzureWebJobs", you can specify only the remainder of the name here. For example, if you set connection to "MyStorage", the Functions runtime looks for an app setting that is named "AzureWebJobsMyStorage." If you leave connection empty, the Functions runtime uses the default Storage connection string in the app setting that is named AzureWebJobsStorage.

Identity-based connections

If you're using version 5.x or higher of the extension, instead of using a connection string with a secret, you can have the app use an Microsoft Entra identity. To use an identity, you define settings under a common prefix that maps to the connection property in the trigger and binding configuration.

If you're setting connection to "AzureWebJobsStorage", see Connecting to host storage with an identity. For all other connections, the extension requires the following properties:

Property Environment variable template Description Example value
Queue Service URI <CONNECTION_NAME_PREFIX>__queueServiceUri1 The data plane URI of the queue service to which you're connecting, using the HTTPS scheme. https://<storage_account_name>.queue.core.windows.net

1 <CONNECTION_NAME_PREFIX>__serviceUri can be used as an alias. If both forms are provided, the queueServiceUri form is used. The serviceUri form can't be used when the overall connection configuration is to be used across blobs, queues, and/or tables.

Other properties may be set to customize the connection. See Common properties for identity-based connections.

When hosted in the Azure Functions service, identity-based connections use a managed identity. The system-assigned identity is used by default, although a user-assigned identity can be specified with the credential and clientID properties. Note that configuring a user-assigned identity with a resource ID is not supported. When run in other contexts, such as local development, your developer identity is used instead, although this can be customized. See Local development with identity-based connections.

Grant permission to the identity

Whatever identity is being used must have permissions to perform the intended actions. For most Azure services, this means you need to assign a role in Azure RBAC, using either built-in or custom roles which provide those permissions.

Important

Some permissions might be exposed by the target service that are not necessary for all contexts. Where possible, adhere to the principle of least privilege, granting the identity only required privileges. For example, if the app only needs to be able to read from a data source, use a role that only has permission to read. It would be inappropriate to assign a role that also allows writing to that service, as this would be excessive permission for a read operation. Similarly, you would want to ensure the role assignment is scoped only over the resources that need to be read.

You will need to create a role assignment that provides access to your queue at runtime. Management roles like Owner are not sufficient. The following table shows built-in roles that are recommended when using the Queue Storage extension in normal operation. Your application may require additional permissions based on the code you write.

Binding type Example built-in roles
Trigger Storage Queue Data Reader, Storage Queue Data Message Processor
Output binding Storage Queue Data Contributor, Storage Queue Data Message Sender

Poison messages

When a queue trigger function fails, Azure Functions retries the function up to five times for a given queue message, including the first try. If all five attempts fail, the functions runtime adds a message to a queue named <originalqueuename>-poison. You can write a function to process messages from the poison queue by logging them or sending a notification that manual attention is needed.

To handle poison messages manually, check the dequeueCount of the queue message.

Peek lock

The peek-lock pattern happens automatically for queue triggers. As messages are dequeued, they are marked as invisible and associated with a 10-minute timeout managed by the Storage service. This timeout can't be changed.

When the function starts, it starts processing a message under the following conditions.

  • If the function is successful, then the function execution completes and the message is deleted.
  • If the function fails, then the message visibility is reset. After being reset, the message is reprocessed the next time the function requests a new message.
  • If the function never completes due to a crash, the message visibility expires and the message re-appears in the queue.

All of the visibility mechanics are handled by the Storage service, not the Functions runtime.

Polling algorithm

The queue trigger implements a random exponential back-off algorithm to reduce the effect of idle-queue polling on storage transaction costs.

The algorithm uses the following logic:

  • When a message is found, the runtime waits 100 milliseconds and then checks for another message
  • When no message is found, it waits about 200 milliseconds before trying again.
  • After subsequent failed attempts to get a queue message, the wait time continues to increase until it reaches the maximum wait time, which defaults to one minute.
  • The maximum wait time is configurable via the maxPollingInterval property in the host.json file.

For local development the maximum polling interval defaults to two seconds.

Note

In regards to billing when hosting function apps in the Consumption plan, you are not charged for time spent polling by the runtime.

Concurrency

When there are multiple queue messages waiting, the queue trigger retrieves a batch of messages and invokes function instances concurrently to process them. By default, the batch size is 16. When the number being processed gets down to 8, the runtime gets another batch and starts processing those messages. So the maximum number of concurrent messages being processed per function on one virtual machine (VM) is 24. This limit applies separately to each queue-triggered function on each VM. If your function app scales out to multiple VMs, each VM will wait for triggers and attempt to run functions. For example, if a function app scales out to 3 VMs, the default maximum number of concurrent instances of one queue-triggered function is 72.

The batch size and the threshold for getting a new batch are configurable in the host.json file. If you want to minimize parallel execution for queue-triggered functions in a function app, you can set the batch size to 1. This setting eliminates concurrency only so long as your function app runs on a single virtual machine (VM).

The queue trigger automatically prevents a function from processing a queue message multiple times simultaneously.

host.json properties

The host.json file contains settings that control queue trigger behavior. See the host.json settings section for details regarding available settings.

Next steps