Deploy NAT Gateway for outbound access from AVD - use the source IP address range (based on the one you assigned to the NAT Gateway) to control the scope of CA policy
If the above response helps answer your question, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.
hth
Marcin