Unable to delete leftover resources (VNet/LB)

Question

I created a private Container App with an App Gateway in front of it. When I tried to delete the stack, some resources remained and became orphaned.

At first, I tried to run the diagnostic tool on the Vnet, where I got:

Resources are out-of-sync with Azure Resource Manager We've discovered that resources related to this issue are out-of-sync with Azure Resource Manager. We have now synced them to resolve the issue. No additional actions are required on your part.

20 minutes later, the Vnet still cannot be deleted. Tried again with the tool, no good. A 3rd time? No luck.

Further investigation finds that the AKS managed resource group (created by ACA) did not delete properly. In particular, the load balancer remained in a AKS managed resource group - a group that I cannot delete (likely because it is managed by Azure). So now I am stuck. Just leaving the resources there is of course not an option since load balancers do cost money to run.

Here are some screenshots to help illustrate my problem:

The managed resource group created by ACA shows some resources left over:

The error I got when i try to manually delete the load balancers:

I do not think I have a way to access the vm scale set or backend pool set by k8s via ACA. Any help will be appreciated!

Edit: after one evening, the resources clean themselves up and I am able to clear everything.