My point is not to have you use CLI. Rather, it is that you can have your application use the same technique that CLI uses. Namely, use Proof Key for Code Exchange (PKCE) to obtain token for user principal, and then use this token to access Azure Key Vault.
Basically your application would listen on localhost ephemeral port for incoming http request, open Azure AD authentication page using default web browser, user signs in to Azure AD, page redirects to localhost, your application uses the returned auth code/client id/etc. to obtain token from AAD, then finally uses token to access key vault.
Microsoft identity platform and OAuth 2.0 authorization code flow
https://learn.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-auth-code-flow