Hi Harry Williams!
Azure Key Vault requires that any entity attempting to interact with the Key Vault service, has an identity registered in the Microsoft Entra ID (previously known as Azure Active Directory) tenant. That entity must also be given access to the data plane of the Key Vault via RBAC role assignment or an access policy, depending on the access configuration of the Key Vault.
If the Key Vault firewall is enabled, the IP of the caller must be added to the allow list on the Key Vault firewall as well.
Here's a link to the Key Vault Developer's guide that you might find useful as well.
I hope this helps! If you need additional assistance, feel free to reply with questions/more details about your environment and goals.
Best, Jessica
Please "Accept the answer" (Yes), and share your feedback if the suggestion answers you’re your query. This will help us and others in the community as well.