How can I add new secrets to Azure keyvault which has private endpoint enabled

Question

I have a keyvault which has private endpoint enabled.

I have allowed our WAN IP on the keyvault firewall but when I browse to the "Secrets" tab then I get the following error:-

" The connection to data plane failed. Please refresh and try again. If Private Links are enabled on the vault and the issue persists please follow the steps in the following link https://go.microsoft.com/fwlink/?linkid=2156688 . "

Also, I have VMs on the same subnet as the keyvault endpoint address. But the internet access is only allowed through proxy on those VMs. The subnets for these VMs are allowed on the keyvault firewall.

If try to access the keyvault from Azure console from these VMs through proxy I get following error:-

"Refresh the browser to try again.

Microsoft_Azure_KeyVault extension failed to load.

Please visit Portal Self Help to perform diagnostics."

There is a DNS problem I have noticed with our Azure hosted DNS zone. "keyvaultname.vaultcore.azure.net" does not alias to "keyvaultname.privatelink.vaultcore.azure.net" on these Azure VMs or on on-prem network. So for now I added a host file entry to resolve "keyvaultname.vaultcore.azure.net" to the private endpoint IP address.

But still the keyvault console access gives the above error on these Azure VMs.

How can I access existing secrets and add new ones to this vault using console (or powershell from the Azure VMs will be the only option to access these)?