Thank you for reaching out and apologies for the delay here.
I understand you have 2 VMs behind an Azure Load balancer and you wish to forward all traffic to VM A until it is up and if it goes down only then forwards traffic to VM B.
AFAIK this is not possible to implement using Azure load balancer as it currently does not support active/passive or weighted load balancing. It will help if you could upvote this features request on the feedback portal.
Although depending on your requirement you can explore the option of using Azure Traffic Manager along with Azure Load Balancer. Something similar to the architecture discussed here for Multitier application.
You can also explore additional load balancing options available in Azure.
Hope this helps. Please let me know if you have any questions. Thank you!