Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Azure HorizonDB is a fully managed database service built on PostgreSQL. It separates compute from storage and supports reader nodes for high availability and read scaling. Zone-redundant storage protects committed data. For clusters in paired regions, you can configure geo-redundant backups to recover to a new cluster in any region where Azure HorizonDB is available after a region-wide failure.
When you use Azure, reliability is a shared responsibility. Microsoft provides a range of capabilities to support resiliency and recovery. You're responsible for understanding how those capabilities work within all of the services you use, and selecting the capabilities you need to meet your business objectives and uptime goals.
This article describes resilience to transient faults, availability zone failures, region-wide failures, and service maintenance in Azure HorizonDB. It also describes backup and restore.
Production deployment recommendations
The Azure Well-Architected Framework provides recommendations across reliability, security, cost, operations, and performance. To understand how these areas influence each other and contribute to a reliable Azure HorizonDB solution, see Architecture best practices for Azure HorizonDB.
Reliability architecture overview
This section describes how the service's architecture supports reliability. It introduces the logical architecture, including the resources and features that you deploy and use, and the physical architecture, including the underlying infrastructure that runs the service.
Logical architecture
The following components determine how your Azure HorizonDB database handles failures:
- Cluster: The top-level resource that contains your databases, pools, compute, and shared storage. You configure cluster-level settings such as backup retention, geo-redundant backup, authentication, and network access.
- Pool: A group of compute nodes with the same compute size and configuration. Each cluster currently has one default pool. You configure the pool's compute size, node count, and high availability option.
- Parameter group: A collection of PostgreSQL configuration settings that you associate with a pool. You choose the settings and association, and Microsoft applies them to the pool's nodes.
- Compute nodes: Microsoft-managed instances of the PostgreSQL engine that process queries and transactions. Each pool has one writer node, which accepts writes and reads, and can have up to 15 reader nodes, which serve reads and act as failover candidates for the writer node.
- Connection endpoints: Service-managed addresses that applications use to connect. The cluster's read-write endpoint routes connections to the current writer node. The pool's read-only endpoint balances connections across reader nodes. When no reader nodes exist, the read-only endpoint routes to the writer node. It doesn't enforce read-only access.
Compute high availability (HA) requires at least one reader node in addition to the writer node. Reader nodes serve read traffic while also providing failover capacity. If the writer node becomes unavailable, Microsoft automatically promotes an available reader node to the writer role and updates routing through the read-write endpoint. Applications reconnect through the same endpoint. The service also uses failover during maintenance.
In Azure HorizonDB documentation, planned failover means a failover that you initiate. Unplanned failover means a service-triggered failover, including automatic failover during an outage or maintenance. The Azure portal labels the customer-initiated operation Failover.
Physical architecture
Azure HorizonDB is a regional service. Microsoft manages the database infrastructure, storage redundancy, failure detection, and compute failover. You're responsible for configuring compute HA, selecting backup settings, and preparing your applications for interruptions and recovery.
All nodes in the default pool use the same compute configuration and share the same underlying data. Adding a reader node doesn't create a separate copy of the database. You choose the pool's capacity and high availability option. Microsoft provisions and operates the nodes.
Storage
Azure HorizonDB separates database processing from data durability. Microsoft manages the following storage components:
- Write-ahead log (WAL) service: Stores the record of database changes. Before the service acknowledges a committed transaction, it makes the transaction's WAL durable across availability zones.
- Shared data storage: Stores database pages and asynchronously applies changes from the WAL. All compute nodes access this shared storage.
Azure Blob Storage provides zone-redundant storage for backups and the WAL archive. Microsoft takes storage snapshots, which capture database state, and continuously archives WAL so that you can restore to a point in time.
Each compute node also has a local cache, which stores frequently accessed data pages on that node's local storage. Caches aren't shared between nodes. After a failover or node replacement, the new node might need to populate its cache before it reaches its usual performance.
Temporary tablespaces hold temporary database objects and intermediate work, such as sorts. Don't store durable application data in temporary tablespaces as their contents don't survive a restart or failover.
Storage is zone-redundant independently of the compute configuration. A cluster with only a writer node protects committed data across zones, but it doesn't have another compute node available for failover. For compute availability during a zone outage, configure zone-redundant high availability.
Management and application dependencies
The control plane handles management operations such as creating clusters, changing configuration, and scaling compute. The control plane is zone-redundant. The data plane handles application connections, queries, and transactions through the connection endpoints. Microsoft operates both planes. You configure resources through the control plane and make your application resilient to data-plane interruptions.
Applications also depend on network connectivity and authentication. If you use private endpoints, which provide private network access to the cluster, you're responsible for the application network connectivity and DNS configuration. If you use Microsoft Entra authentication, access also depends on Microsoft Entra ID. Database recovery alone doesn't recover these application dependencies.
For more information, see Azure HorizonDB overview and Compute nodes in Azure HorizonDB.
Resilience to transient faults
Transient faults are short, intermittent failures in components. They occur frequently in a distributed environment like the cloud, and they're a normal part of operations. Transient faults correct themselves after a short period of time. It's important that your applications can handle transient faults, usually by retrying affected requests.
All cloud-hosted applications should follow the Azure transient fault handling guidance when they communicate with any cloud-hosted APIs, databases, and other components. For more information, see Recommendations for handling transient faults.
Azure HorizonDB manages database failover and maintenance as part of normal service operations. These activities can briefly interrupt application connections. You're responsible for configuring your PostgreSQL client or application to reconnect and handle transient connection errors.
Use the read-write and read-only endpoints to connect to Azure HorizonDB. During writer failover, the service updates routing through the read-write endpoint to the new writer node. Existing connections to the previous writer drop, so applications need to reconnect. The endpoint address stays the same, so you don't need to change your connection strings.
Configure retries with exponential backoff and a limit on retry attempts. Ensure that retrying an operation doesn't duplicate a business action. A dropped connection can prevent your application from receiving a transaction's result, so don't assume that every interrupted request failed before committing. For general retry guidance, see the Retry pattern.
Uncommitted transactions roll back during failover. Your application needs to retry the whole transaction, not just the interrupted statement. Keep transactions short. Long-running transactions can delay failover during maintenance and increase the amount of work that rolls back when a failure triggers failover. For more information, see High availability and failover in Azure HorizonDB and Perform failover in Azure HorizonDB.
Resilience to availability zone failures
Availability zones are physically separate groups of datacenters within an Azure region. When one zone fails, services can fail over to one of the remaining zones.
Azure HorizonDB clusters separate storage from compute. The two layers provide different protection against availability zone failures:
Storage: The storage layer is zone-redundant for every cluster, regardless of the compute configuration. Zone-redundant storage protects committed data, but it doesn't by itself keep a cluster available during a zone failure.
Compute: For compute zone redundancy, configure at least one reader node in addition to the writer node and set the pool's high availability option to
ZoneRedundant.The high availability option determines compute placement:
- ZoneRedundant: Microsoft distributes compute nodes across at least two availability zones.
- AnyZone: Microsoft prioritizes provisioning success. The cluster attempts to distribute compute nodes across multiple availability zones but can place all nodes in one zone if capacity is unavailable. This option doesn't guarantee compute availability during a zone outage.
- Disabled: The pool has a single node and no compute failover target. This mode provides no compute protection against zone outages.
The Azure HorizonDB control plane is also zone-redundant.
Requirements
- Region support: All Azure HorizonDB regions support availability zones and zone redundancy.
- High availability mode: Set the pool's high availability option to
ZoneRedundant. - Node count: Configure at least two nodes in the default pool: one writer node and at least one reader node.
Instance distribution across zones
Storage: Azure HorizonDB uses zone-redundant storage (ZRS), which replicates data across at least three availability zones. For more information, see Zone-redundant storage.
Compute: Microsoft selects the zones for each compute node based on the pool's HA mode. You don't specify availability zones or explicitly configure zone placement for each node.
With the ZoneRedundant HA mode Microsoft places nodes across at least two availability zones. With AnyZone, Microsoft attempts to place nodes across two availability zones but can fall back to a single zone if capacity is unavailable.
Cost
There's no additional charge for zone redundancy or the HA mode you select. For the same node count and compute size, ZoneRedundant and AnyZone cost the same. For pricing details, see Azure HorizonDB pricing.
Configure availability zone support
Create a new zone-redundant cluster: Configure the default pool with at least one reader node and select the
ZoneRedundanthigh availability option. For instructions, see Configure high availability in Azure HorizonDB.Enable zone redundancy on an existing cluster: Add a reader node if the pool doesn't already have one, and select the
ZoneRedundanthigh availability option. Adding a reader node doesn't require downtime for the existing nodes, because the new node uses the shared storage rather than receiving a database copy. For instructions, see Configure high availability in Azure HorizonDB.Changing an existing pool from
AnyZonetoZoneRedundantcan replace a reader node with one in another zone. Connections to the replaced node are disconnected, so applications need to handle transient faults. For this behavior, see Pools - Update.
Note
The ZoneRedundant HA mode requires compute capacity in at least two availability zones, so provisioning can take longer than with AnyZone. If Azure HorizonDB can't allocate all requested nodes or satisfy the required zone distribution, the configuration operation fails and preserves the previous resource state.
Capacity planning and management
During a zone outage, nodes in the affected zone become unavailable, which reduces the pool's overall compute capacity. Plan for the surviving nodes to handle the workload. For example, if four nodes are distributed evenly across two zones, plan for two surviving nodes after a zone outage: one writer and one reader. Validate that these nodes can handle your required write and read workload under a representative load.
If your workload can't tolerate reduced capacity, you can overprovision compute before an outage so that the surviving nodes can handle the required load. Choose the pool's compute size and node count so that a promoted reader node can handle the writer workload and the remaining reader nodes can handle the required read workload.
You can change the pool's vCore allocation without a noticeable application interruption. However, don't rely on scaling during an outage to provide the capacity that the workload needs immediately after failover. For scaling instructions, see Scale compute in Azure HorizonDB.
If zone placement requirements prevent replacement nodes from being provisioned during an outage, you can temporarily change the pool's HA mode to AnyZone to allow provisioning in an available zone. This trades guaranteed compute zone redundancy for the ability to restore compute capacity. After the outage, change the mode back to ZoneRedundant when capacity is available across zones. The change can replace a reader node and interrupt its connections, as described in Configure availability zone support.
Microsoft attempts to distribute compute nodes evenly across zones, but actual distribution can vary, so plan for the loss of the nodes in the zone with the most nodes rather than assuming exactly 50% capacity loss.
Behavior when all zones are healthy
This section describes what to expect when you configure an Azure HorizonDB pool for zone redundancy and all zones are operational.
Cross-zone operation: For compute nodes, the read-write endpoint routes connections to the writer node, whichever zone it's in. The read-only endpoint balances connections across reader nodes regardless of their availability zones.
For storage, the compute nodes in different availability zones access the same shared zone-redundant storage.
Cross-zone data replication: The WAL service makes committed transactions durable across zones before acknowledging them. The shared data storage applies WAL changes. Reader nodes can lag behind the writer node, so send reads that require the latest committed data to the read-write endpoint.
Behavior during a zone failure
This section describes what to expect when you configure an Azure HorizonDB pool for zone redundancy and one of the zones it uses has an outage.
Detection and response: A zone outage can make compute nodes in that zone unavailable. Microsoft is responsible for detecting affected nodes through node health signals and automatically responding according to their roles:
- Writer node affected: The service isolates the failed writer to prevent further writes, selects an available reader node in a healthy zone whose applied WAL position is closest to the writer's, and promotes it. After the promoted node accepts writes, the service updates the read-write endpoint.
- Only reader nodes affected: The writer node doesn't need to fail over. Reader nodes in healthy zones continue to serve reads, but the pool has less read capacity.
Notification: Microsoft doesn't automatically notify you when a zone is down. However, you can use Azure Service Health to understand the overall health of the service, including any zone failures, and you can set up Service Health alerts to notify you of problems.
For node availability, configure an Azure Monitor alert on the Postgres Instance Is Alive metric, split by node. You're responsible for routing alerts to your operations team.
Active requests: Connections to nodes in the affected zone drop.
- Writer node affected: Uncommitted transactions roll back. Your application reconnects through the read-write endpoint after failover.
- Reader nodes affected: Reads in progress on those nodes are interrupted. Your application reconnects through the read-only endpoint, which routes connections to available nodes.
Handle interrupted requests as described in Resilience to transient faults.
Expected data loss: Committed transactions aren't lost during compute failover. The promoted node accesses the same durable WAL through the shared storage. However, this protection doesn't preserve uncommitted transactions or temporary data.
Expected downtime: The downtime depends on which nodes were affected by the zone outage.
- Writer node affected: Writes are interrupted while Microsoft detects the failure, isolates the failed writer, selects an available reader node in a healthy zone whose applied WAL position is closest to the writer's, promotes it, waits until it accepts writes, and updates the read-write endpoint. Failover typically takes a few seconds. Shared storage avoids the need to copy the database to the promoted node.
- Only reader nodes affected: No writer failover is required. Reads on affected nodes are interrupted until applications reconnect to available nodes. Reduced read capacity can also affect query performance.
The duration of application interruptions also depends on reconnection and retry behavior. For more information, see Resilience to transient faults.
Redistribution: If the writer node is affected, an available reader node takes over the writer role. The remaining reader nodes provide the pool's read capacity, subject to the nodes and zones that remain available.
Microsoft attempts to replace failed nodes. With the ZoneRedundant HA mode, node replacement might be delayed until capacity is available to satisfy the zone distribution requirements. You can temporarily switch the cluster to the AnyZone HA mode to allow provisioning in an available zone, with the tradeoffs described in Capacity planning and management.
Zone recovery
Microsoft manages node recovery, and recovered nodes automatically rejoin the pool. You don't need to take action to recover nodes. You're responsible for checking application connectivity, workload performance, and available capacity after the zone recovers.
Test for zone failures
You can initiate a planned failover to test whether applications reconnect through the read-write endpoint and handle interrupted transactions.
Run disruptive tests in an environment where you can tolerate the interruption. Measure downtime from the application's perspective rather than using the total failover operation time reported in the Azure portal. Also measure performance after reconnection under representative load. For instructions, see Perform failover in Azure HorizonDB.
A failover that you initiate doesn't reproduce every effect of a zone outage, such as loss of other application components, failures in shared storage, or failure-detection delays.
Resilience to region-wide failures
Azure HorizonDB clusters run in a single Azure region. If that region becomes unavailable, the cluster is unavailable.
If your cluster is in a paired region and you enable geo-redundant backups, you can restore those backups to a new cluster in any region where Azure HorizonDB is available, not only the paired region. Recovery involves downtime while you restore the cluster and update application connections. It can also involve data loss because backup copies reach the paired region with a delay, so the latest available backup might not include the most recent changes. For more information, see Geo-redundant backup.
Custom multiregion solutions for resiliency
If your recovery objectives require an approach other than restoring from geo-redundant backups, you're responsible for designing and operating a custom recovery solution.
For data export and import options, see Azure Storage extension in Azure HorizonDB. You can also use PostgreSQL tools such as pg_dump to export data. Data exports aren't full cluster backups, so validate that you can recover the required data and configuration.
Backup and restore
Zone redundancy and compute failover don't replace backups. They protect against infrastructure failures, but they don't reverse a data change that your application commits. Backups provide a recovery path for data corruption, accidental data deletion, and cluster failures. For more information, see Redundancy, replication, and backup.
Microsoft automatically backs up your cluster by creating storage snapshots several times a day and continuously archiving WAL. Together, the snapshots and archived WAL let you restore your database to a point in time. Backup operations run in the storage layer rather than on the compute nodes, leaving the compute nodes available to serve application traffic. Backup storage is zone-redundant, and you can select geo-redundant backup when creating a cluster in a supported paired region.
Retention: You configure backup retention from 1 to 35 days. If you omit the retention setting, the service uses seven days. For retention longer than 35 days, separately export the required data to storage that you manage. An export isn't equivalent to a full cluster backup, so validate that you can recover the required data and configuration.
Restore: Same-region point-in-time restore combines a snapshot with archived WAL to recover database state within the retention period. The time needed for restoration depends on factors such as the database size, the time of the most recent backup, and the amount of WAL that must be processed during recovery.
Restoration creates a new cluster. It doesn't overwrite the existing cluster. You're responsible for selecting the recovery point, verifying the recovered data, and updating application connections. To recover from an accidental data change, restore to a point before the change and retrieve the data that you need from the restored cluster.
Cluster configuration: A restore operation doesn't recreate pools or other child resources. You're responsible for configuring the restored cluster's compute, networking, and authentication settings. The Azure portal can apply some defaults, so verify the resulting configuration before connecting your application. For details, see the REST API reference.
Keep your cluster configuration in infrastructure as code (IaC) so that you can recreate settings that the restore operation doesn't preserve.
Test restores regularly and select retention based on how long an error might remain undetected.
For more information, see Backups in Azure HorizonDB and Restore for Azure HorizonDB.
Geo-redundant backup
In paired regions, you can enable geo-redundant backup storage when you create a cluster. You can't enable it on an existing cluster. Microsoft manages backup creation and copies backups to the paired region. You can restore these backups to a new cluster in any region where Azure HorizonDB is available. Backups don't provide running compute capacity or an additional database endpoint. For more information, see Geo-redundant backup and restore in Azure HorizonDB.
Geo-redundant backups aren't available for clusters created regions that don't have a paired region. Geo-redundant backups also aren't supported for clusters encrypted with customer-managed keys.
You initiate geo-restore to create a new cluster in any region where Azure HorizonDB is available, using the latest available backup copy stored in the paired region. The destination doesn't have to be the paired region. You can't select a historical restore point for geo-restore. Backups reach the paired region with a delay, so the restored cluster might not include the most recent changes. For instructions, see Geo-restore in Azure HorizonDB.
You're responsible for preparing compute capacity, regional quotas, and network access for the restored cluster, verifying the recovered data, and updating application connections to its new endpoints. Include recovery of application dependencies in your plan. Test geo-restore and measure data loss and end-to-end recovery time against your workload's recovery objectives.
After recovery, you're responsible for choosing when to return to the original region and preventing applications from writing independently to both clusters. Recovery doesn't automatically merge their changes.
Geo-redundant backup storage increases backup storage costs. The restored cluster also incurs charges for the resources that you deploy. For more information, see Azure HorizonDB pricing and Business continuity in Azure HorizonDB.
Resilience to cluster deletion
Use Azure role-based access control (Azure RBAC) to restrict who can delete production clusters.
You can also apply a Delete resource lock to prevent cluster deletion through the control plane. Resource locks don't prevent destructive SQL operations such as deleting rows or tables, and they aren't a substitute for backups. For configuration instructions, see Protect against accidental deletion by using resource locks in Azure HorizonDB.
You can't currently restore a deleted cluster from its retained backups.
Resilience to service maintenance
Maintenance keeps your cluster secure and up to date through service updates, security patches, and database minor-version updates that Microsoft applies. Microsoft schedules maintenance by using a system-managed maintenance window.
Successful scheduled maintenance events are normally at least 30 days apart. Critical updates, such as urgent security patches, can occur sooner. Maintenance notifications aren't available.
When the pool has reader nodes, the service automatically triggers failover so that a reader node takes over the writer role during maintenance. Shared storage avoids copying the database or rewinding WAL before the promoted node can operate. Applications might experience connection interruptions and need to reconnect through the read-write endpoint. Follow the guidance in Resilience to transient faults. Keep transactions short to reduce delays during failover.
If the pool has no reader nodes, maintenance that requires restarting the writer interrupts both reads and writes until the writer becomes available again.
Avoid resource configuration changes while maintenance is in progress.
For maintenance settings and notifications, see Scheduled maintenance in Azure HorizonDB.