Respond to application failures
Previous sections have focused on automated recovery strategies that are critical for high availability. However, sometimes manual intervention is needed.
Alerts. Monitor your application for warning signs that proactive intervention is needed. For example, if you see that SQL Database or Cosmos DB consistently throttles your application, you might need to increase your database capacity or optimize your queries. In this example, even though the application might handle the throttling errors transparently, your telemetry should still raise an alert so that you can follow up. It's recommended to configure alerts on Azure resources metrics and diagnostics logs against the services limits and quotas thresholds. We recommend configuring alerts on metrics as metrics are lower latency compared with diagnostics logs. In addition, Azure is able to provide with some out-of-the-box health status through resource health that can help diagnose throttling of Azure services.
Failover. Configure a disaster recovery strategy in your application. The appropriate strategy depends on your SLAs. For most scenarios, an active-passive implementation is sufficient. Most Azure services support either manual or automated failover. For example, in an IaaS application, use Azure Site Recovery for the web and logic tiers and SQL Always On Availability Groups for the database tier. Traffic Manager provides automated failover across regions.
Operational readiness testing. Perform an operational readiness test for both failover to the secondary region and failback to the primary region. Many Azure services support manual failover or test failover for disaster recovery drills. Alternatively, you can simulate an outage by shutting down or removing services.
Data consistency check. If a failure happens in a data store, there may be data inconsistencies when the store becomes available again, especially if the data was replicated. For Azure services that provide cross-regional replication, look at the RTO and RPO to understand the expected data loss in a failure. Review the SLAs for Azure services to understand whether cross-regional failover can be initiated manually or is initiated by Microsoft. For some services, Microsoft decides when to perform the failover. Microsoft might prioritize the recovery of data in the primary region, only failing over to a secondary region if data in the primary region is deemed unrecoverable. For example, Geo-redundant storage and Key Vault follow this model.
Restoring from backup. In case of accidental deletion or corruption of data, it is required to revert the data to a previous known state using backups. Azure Backup helps restore the Azure Virtual Machines quickly from recent backups using instant restore feature thus eliminating need to transfer backup from vault storage to customer's subscription there by lowering RTO. It also offers various restore choices like restoring to create new virtual machine or restore the disks or just specific file/folder that fits different needs. By configuring backups on Azure Blobs, Azure Disks, Azure File Shares, you also get the ability to restore data stored by your application on various storage solutions on Azure. Databases are critical to many applications. With Azure Backup, you can gain point in time recovery of the SQL Server and SAP HANA databases running on Azure Virtual Machines providing RPO as low as few minutes.