Got it. It's Update-MgApplication with the -KeyCredentials parameter for a new cert, or Add-MgApplicationKey to update an existing cert. The word "Key" through me off since you aren't actually uploading a private key, but I guess it is meant to refer to the fact that the authentication will use a key (which will be validated by the cert you upload).
Here's a working example of a function that pulls the cert from a keyvault and uploads to a registered app:
function Set-AppCredential
{
Param(
[Parameter(Mandatory)]
[string]$AppDisplayName,
[Parameter(Mandatory)]
[string]$KeyVaultName,
[Parameter(Mandatory)]
[string]$CertificateName
)
$Application = Get-MgApplication -Filter "DisplayName eq '$($AppDisplayName)'"
$KeyVaultCertificate = Get-AzKeyVaultCertificate -VaultName $KeyVaultName -Name $CertificateName
$CertCredential = @{
Type = "AsymmetricX509Cert"
Usage = "Verify"
Key = $KeyVaultCertificate.Certificate.RawData
}
Update-MgApplication -ApplicationId $Application.Id -KeyCredentials @($CertCredential)
}