Durable Functions storage providers

บทความ
03/03/2023

Durable Functions is a set of Azure Functions triggers and bindings that are internally powered by the Durable Task Framework (DTFx). DTFx supports various backend storage providers, including the Azure Storage provider used by Durable Functions. Starting in Durable Functions v2.5.0, users can configure their function apps to use DTFx storage providers other than the Azure Storage provider.

Note

For many function apps, the default Azure Storage provider for Durable Functions is likely to suffice, and is the easiest to use since it requires no extra configuration. However, there are cost, scalability, and data management tradeoffs that may favor the use of an alternate storage provider.

Two alternate storage providers were developed for use with Durable Functions and the Durable Task Framework, namely the Netherite storage provider and the Microsoft SQL Server (MSSQL) storage provider. This article describes all three supported providers, compares them against each other, and provides basic information about how to get started using them.

Note

It's not currently possible to migrate data from one storage provider to another. If you want to use a new storage provider, you should create a new app configured with the new storage provider.

Azure Storage

Azure Storage is the default storage provider for Durable Functions. It uses queues, tables, and blobs to persist orchestration and entity state. It also uses blobs and blob leases to manage partitions. In many cases, the storage account used to store Durable Functions runtime state is the same as the default storage account used by Azure Functions (AzureWebJobsStorage). However, it's also possible to configure Durable Functions with a separate storage account. The Azure Storage provider is built-into the Durable Functions extension and doesn't have any other dependencies.

The key benefits of the Azure Storage provider include:

No setup required - you can use the storage account that was created for you by the function app setup experience.
Lowest-cost serverless billing model - Azure Storage has a consumption-based pricing model based entirely on usage (more information).
Best tooling support - Azure Storage offers cross-platform local emulation and integrates with Visual Studio, Visual Studio Code, and the Azure Functions Core Tools.
Most mature - Azure Storage was the original and most battle-tested storage backend for Durable Functions.
Support for using identity instead of secrets for connecting to the storage provider.

The source code for the DTFx components of the Azure Storage storage provider can be found in the Azure/durabletask GitHub repo.

Note

Standard general purpose Azure Storage accounts are required when using the Azure Storage provider. All other storage account types are not supported. We highly recommend using legacy v1 general purpose storage accounts because the newer v2 storage accounts can be significantly more expensive for Durable Functions workloads. For more information on Azure Storage account types, see the Storage account overview documentation.

Netherite

The Netherite storage backend was designed and developed by Microsoft Research. It uses Azure Event Hubs and the FASTER database technology on top of Azure Page Blobs. The design of Netherite enables significantly higher-throughput processing of orchestrations and entities compared to other providers. In some benchmark scenarios, throughput was shown to increase by more than an order of magnitude when compared to the default Azure Storage provider.

The key benefits of the Netherite storage provider include:

Significantly higher throughput at lower cost compared to other storage providers.
Supports price-performance optimization, allowing you to scale-up performance as-needed.
Supports up to 32 data partitions with Event Hubs Basic and Standard SKUs.
More cost-effective than other providers for high-throughput workloads.

You can learn more about the technical details of the Netherite storage provider, including how to get started using it, in the Netherite documentation. The source code for the Netherite storage provider can be found in the microsoft/durabletask-netherite GitHub repo. A more in-depth evaluation of the Netherite storage provider is also available in the following research paper: Serverless Workflows with Durable Functions and Netherite.

Note

The Netherite name originates from the world of Minecraft.

Microsoft SQL Server (MSSQL)

The Microsoft SQL Server (MSSQL) storage provider persists all state into a Microsoft SQL Server database. It's compatible with both on-premises and cloud-hosted deployments of SQL Server, including Azure SQL Database.

The key benefits of the MSSQL storage provider include:

Supports disconnected environments - no Azure connectivity is required when using a SQL Server installation.
Portable across multiple environments and clouds, including Azure-hosted and on-premises.
Strong data consistency, enabling backup/restore and failover without data loss.
Native support for custom data encryption (a feature of SQL Server).
Integrates with existing database applications via built-in stored procedures.

You can learn more about the technical details of the MSSQL storage provider, including how to get started using it, in the Microsoft SQL provider documentation. The source code for the MSSQL storage provider can be found in the microsoft/durabletask-mssql GitHub repo.

Configuring alternate storage providers

Configuring alternate storage providers is generally a two-step process:

Add the appropriate NuGet package to your function app (this requirement is temporary for apps using extension bundles).
Update the host.json file to specify which storage provider you want to use.

If no storage provider is explicitly configured in host.json, the Azure Storage provider will be enabled by default.

Configuring the Azure storage provider

The Azure Storage provider is the default storage provider and doesn't require any explicit configuration, NuGet package references, or extension bundle references. You can find the full set of host.json configuration options here, under the extensions/durableTask/storageProvider path.

Connections

The connectionName property in host.json is a reference to environment configuration which specifies how the app should connect to Azure Storage. It may specify:

The name of an application setting containing a connection string. To obtain a connection string, follow the steps shown at Manage storage account access keys.
The name of a shared prefix for multiple application settings, together defining an identity-based connection.

If the configured value is both an exact match for a single setting and a prefix match for other settings, the exact match is used. If no value is specified in host.json, the default value is "AzureWebJobsStorage".

Identity-based connections

If you are using version 2.7.0 or higher of the extension and the Azure storage provider, instead of using a connection string with a secret, you can have the app use an Microsoft Entra identity. To do this, you would define settings under a common prefix which maps to the connectionName property in the trigger and binding configuration.

To use an identity-based connection for Durable Functions, configure the following app settings:

Property	Environment variable template	Description	Example value
Blob service URI	`<CONNECTION_NAME_PREFIX>__blobServiceUri`	The data plane URI of the blob service of the storage account, using the HTTPS scheme.	https://<storage_account_name>.blob.core.windows.net
Queue service URI	`<CONNECTION_NAME_PREFIX>__queueServiceUri`	The data plane URI of the queue service of the storage account, using the HTTPS scheme.	https://<storage_account_name>.queue.core.windows.net
Table service URI	`<CONNECTION_NAME_PREFIX>__tableServiceUri`	The data plane URI of a table service of the storage account, using the HTTPS scheme.	https://<storage_account_name>.table.core.windows.net

Additional properties may be set to customize the connection. See Common properties for identity-based connections.

When hosted in the Azure Functions service, identity-based connections use a managed identity. The system-assigned identity is used by default, although a user-assigned identity can be specified with the credential and clientID properties. Note that configuring a user-assigned identity with a resource ID is not supported. When run in other contexts, such as local development, your developer identity is used instead, although this can be customized. See Local development with identity-based connections.

Grant permission to the identity

Whatever identity is being used must have permissions to perform the intended actions. For most Azure services, this means you need to assign a role in Azure RBAC, using either built-in or custom roles which provide those permissions.

Important

Some permissions might be exposed by the target service that are not necessary for all contexts. Where possible, adhere to the principle of least privilege, granting the identity only required privileges. For example, if the app only needs to be able to read from a data source, use a role that only has permission to read. It would be inappropriate to assign a role that also allows writing to that service, as this would be excessive permission for a read operation. Similarly, you would want to ensure the role assignment is scoped only over the resources that need to be read.

You'll need to create a role assignment that provides access to Azure storage at runtime. Management roles like Owner aren't sufficient. The following built-in roles are recommended when using the Durable Functions extension in normal operation:

Your application may require more permissions based on the code you write. If you're using the default behavior or explicitly setting connectionName to "AzureWebJobsStorage", see Connecting to host storage with an identity for other permission considerations.

Configuring the Netherite storage provider

Enabling the Netherite storage provider requires a configuration change in your host.json. For C# users, it also requires an additional installation step.

`host.json` Configuration

The following host.json example shows the minimum configuration required to enable the Netherite storage provider.

{
  "version": "2.0",
  "extensions": {
    "durableTask": {
      "storageProvider": {
        "type": "Netherite",
        "storageConnectionName": "AzureWebJobsStorage",
        "eventHubsConnectionName": "EventHubsConnection"
      }
    }
  }
}

For more detailed setup instructions, see the Netherite getting started documentation.

Install the Netherite extension (.NET only)

Note

If your app uses Extension Bundles, you should ignore this section as Extension Bundles removes the need for manual Extension management.

You'll need to install the latest version of the Netherite Extension on NuGet. This usually means including a reference to it in your .csproj file and building the project.

The Extension package to install depends on the .NET worker you are using:

For the in-process .NET worker, install Microsoft.Azure.DurableTask.Netherite.AzureFunctions.
For the isolated .NET worker, install Microsoft.Azure.Functions.Worker.Extensions.DurableTask.Netherite.

Configuring the MSSQL storage provider

Enabling the MSSQL storage provider requires a configuration change in your host.json. For C# users, it also requires an additional installation step.

`host.json` Configuration

The following example shows the minimum configuration required to enable the MSSQL storage provider.

{
  "version": "2.0",
  "extensions": {
    "durableTask": {
      "storageProvider": {
        "type": "mssql",
        "connectionStringName": "SQLDB_Connection"
      }
    }
  }
}

For more detailed setup instructions, see the MSSQL provider's getting started documentation.

Install the Durable Task MSSQL extension (.NET only)

Note

If your app uses Extension Bundles, you should ignore this section as Extension Bundles removes the need for manual Extension management.

You'll need to install the latest version of the MSSQL storage provider Extension on NuGet. This usually means including a reference to it in your .csproj file and building the project.

The Extension package to install depends on the .NET worker you are using:

For the in-process .NET worker, install Microsoft.DurableTask.SqlServer.AzureFunctions.
For the isolated .NET worker, install Microsoft.Azure.Functions.Worker.Extensions.DurableTask.SqlServer.

Comparing storage providers

There are many significant tradeoffs between the various supported storage providers. The following table can be used to help you understand these tradeoffs and decide which storage provider is best for your needs.

Storage provider	Azure Storage	Netherite	MSSQL
Official support status	✅ Generally available (GA)	✅ Generally available (GA)	✅ Generally available (GA)
External dependencies	Azure Storage account (general purpose v1)	Azure Event Hubs Azure Storage account (general purpose)	SQL Server 2019 or Azure SQL Database
Local development and emulation options	Azurite v3.12+ (cross platform)	Supports in-memory emulation of task hubs (more information)	SQL Server Developer Edition (supports Windows, Linux, and Docker containers)
Task hub configuration	Explicit	Explicit	Implicit by default (more information)
Maximum throughput	Moderate	Very high	Moderate
Maximum orchestration/entity scale-out (nodes)	16	32	N/A
Maximum activity scale-out (nodes)	N/A	32	N/A
KEDA 2.0 scaling support (more information)	❌ Not supported	❌ Not supported	✅ Supported using the MSSQL scaler (more information)
Support for extension bundles (recommended for non-.NET apps)	✅ Fully supported	✅ Fully supported	✅ Fully supported
Price-performance configurable?	❌ No	✅ Yes (Event Hubs TUs and CUs)	✅ Yes (SQL vCPUs)
Disconnected environment support	❌ Azure connectivity required	❌ Azure connectivity required	✅ Fully supported
Identity-based connections	✅ Fully supported	❌ Not supported	⚠️ Requires runtime-driven scaling

Next steps

Learn about Durable Functions performance and scale

แชร์ผ่าน

Durable Functions storage providers

Azure Storage

Netherite

Microsoft SQL Server (MSSQL)

Configuring alternate storage providers

Configuring the Azure storage provider

Connections

Identity-based connections

Grant permission to the identity

Configuring the Netherite storage provider

`host.json` Configuration

Install the Netherite extension (.NET only)

Configuring the MSSQL storage provider

`host.json` Configuration

Install the Durable Task MSSQL extension (.NET only)

Comparing storage providers

Next steps

คำติชม

แหล่งทรัพยากรเพิ่มเติม

แชร์ผ่าน

Durable Functions storage providers

Azure Storage

Netherite

Microsoft SQL Server (MSSQL)

Configuring alternate storage providers

Configuring the Azure storage provider

Connections

Identity-based connections

Grant permission to the identity

Configuring the Netherite storage provider

host.json Configuration

Install the Netherite extension (.NET only)

Configuring the MSSQL storage provider

host.json Configuration

Install the Durable Task MSSQL extension (.NET only)

Comparing storage providers

Next steps

คำติชม

แหล่งทรัพยากรเพิ่มเติม

`host.json` Configuration

`host.json` Configuration