Thank you for reaching out.
If I understand it correctly you wish to limit the access to your static web app so that it can only access via client who are connected to VPN.
I think the solution here will be to use a private endpoint to restrict access to your static web app so that it is only accessible from your private network. Configuring Static Web Apps with a private endpoint allows you to use a private IP address from your VNet. Once this link is created, your static web app is integrated into your VNet. As a result, your static web app is no longer available to the public internet, and is only accessible from machines within your Azure VNet. Similar to what described here.
For DNS resolution as your web app needs to resolve on the on-prem network as well (connected via VPN) you will need to configure a Private DNS Zones as described in the link above and also deploy a Azure Private DNS Resolver or DNS forwarder VM in order to receive and resolve on-prem DNS queries. Both these scenarios are well explained here.
Additional references
https://learn.microsoft.com/en-us/azure/app-service/networking/private-endpoint
Hope this helps! Please let me know if you have any questions. Thank you!
Please "Accept the answer" if the information helped you. This will help us and others in the community as well.