Hi,
Key vault have limitation.
Azure Key Vault service limits
https://learn.microsoft.com/en-us/azure/key-vault/general/service-limits
For example:
For RSA 2,048-bit software keys, 2,000 GET transactions per 10 seconds are allowed.
For RSA 2,048-bit HSM-keys, 1,000 GET transactions per 10 seconds are allowed.
Bulk load if you get each time a secret in the vault shouldnt be working well.
Fault a designed for application side not for user side.
That's might help.
Ken