Hello, @David C !
Are availability sets a relevant feature if we do our patching manually on Azure SQL Server VMs?
Yes. Always On availability groups on Azure Virtual Machines still need to take into consideration redundancy provided by availability sets or availability zones:
Always On availability group on SQL Server on Azure VMs
To increase redundancy and high availability, SQL Server VMs should either be in the same availability set, or different availability zones. Placing a set of VMs in the same availability set protects from outages within a data center caused by equipment failure (VMs within an Availability Set do not share resources) or from updates (VMs within an availability set are not updated at the same time). Availability Zones protect against the failure of an entire data center, with each Zone representing a set of data centers within a region. By ensuring resources are placed in different Availability Zones, no data center-level outage can take all of your VMs offline.
So even if we ignore the benefits of update domains, fault domains provide redundancy protection against transient hardware failures and network or power interruptions.
Additional reading: