Hi,
1.Per description of RDS upgrade guide, the downtime is required when upgrading Connection Brokers.
"The deployment will not be available during RD Connection Broker servers upgrade."
2.When there is Connection Broker High Availability, the upgrade sequence shall be:
-Remove all but one server from the deployment and perform an in-place upgrade.
-Perform upgrades on the remaining RD Connection Broker servers offline and then re-add them to the deployment
Reference link:
Upgrading your Remote Desktop Services deployments to Windows Server 2016
https://learn.microsoft.com/en-us/windows-server/remote/remote-desktop-services/upgrade-to-rds
If the Answer is helpful, please click Accept Answer and upvote it. Thanks.
Thanks,
Jenny