Hello @Huhn, Alexander
The error "Client address is not authorized and caller is not a trusted service in Azure" shows that your client IP address is not authorized.
In Azure KeyVault Settings you can allow access to KeyVault from "All networks" and not "Selected networks"
If you really need to limit access to KeyVault from Azure App Service, integrate Azure App Service into Virtual network and link KeyVault using Service or Private Endpoint.
https://learn.microsoft.com/en-us/azure/key-vault/general/private-link-service?tabs=portal
https://learn.microsoft.com/en-us/azure/key-vault/general/overview-vnet-service-endpoints