API Management root ca invalid using keyvault certificate through ARM but valid manually uploaded

Declan O'Malley 6

Hi All,

I have an issue with APIM and root ca.

my APIM connects to appgateway over HTTPS using a self-signed certificate in my dev environment. APIM has a root ca selfsigned.mydomain.co.uk uploaded through the portal and a client certificate referencing a key vault certificate with a global policy in place to use the MTLS certificate

Appgateway is configured with a server certificate appgateway.mydomain.co.uk which is signed with a root ca called selfsigned.mydomain.co.uk . Appgateway is also configured with MTLS using another certificate appgateway-mtls.mydomain.co.uk which has the certificate chain.

this works! no problems at all. however, if I upload selfsigned.mydomain.co.uk to keyvault as a certificate, supplying the certificate password for the private key and then through my bicep template pull down the secret and assign the root ca to APIM this does not work. i get a SSL/TLS negotiation issue when connecting to app gateway.

The certificate panel in the portal displays the correct , and same, thumbprint as when manually uploading and the deployment is successful. it does take 30 mins to apply the certificate though. I am not supplying the password for the cert as the password is removed in keyvault an

example bicep. the secret appears to be base64 encoded when retrieved from keyvault so no manipulation of the object happens

@secure()

param rootCertificateEncoded string





resource apiManagementService 'Microsoft.ApiManagement/service@2021-08-01' = {

  name: apiManagementServiceName

  location: rgLocation

  tags: tags

  sku: {

    name: sku

    capacity: skuCount

  }

  properties: {

    publisherEmail: publisherEmail

    publisherName: publisherName

    publicNetworkAccess: environment == 'uat' || environment == 'prod' ? 'Disabled' : 'Enabled'

    certificates:[

      {

        storeName:'Root'

        certificate:{

          expiry: '2024-03-07T15:52:28+00:00'

          subject: 'CN=selfsigned.mydomain.co.uk, O=Some Organisation, C=CH'

          thumbprint: 'A5459F3E2FAEA09CFF56DF6BB86E7092CEB4B412'

        }

        encodedCertificate:rootCertificateEncoded

      }

    ]

  }

}

the runs successfully but then fails to connect to appgateway. I'm not sure what else i can try as the thumbprint is correct and APIM reports no errors. if i delete the certificate and upload manually through the portal with the same certificate it works. any suggestions?

Declan O'Malley 6

just wanted to add to this that I uploaded the certificate as a "secret" instead of a "certificate" , in keyvault, by base64 encoding the certificate and supplying the password in the bicep template. This works! however this feels like a bug with APIM not working with a passwordless PFX? is this by design?

example bicep


@secure()
param rootCertificateEncoded string

@secure()
param rootCertificateEncodedPwd string

resource apiManagementService 'Microsoft.ApiManagement/service@2021-08-01' = {
  name: apiManagementServiceName
  location: rgLocation
  tags: tags
  sku: {
    name: sku
    capacity: skuCount
  }
  properties: {
    publisherEmail: publisherEmail
    publisherName: publisherName
    publicNetworkAccess: environment == 'uat' || environment == 'prod' ? 'Disabled' : 'Enabled'
    certificates:[
      {
        storeName:'Root'
        certificate:{
          expiry: '2024-03-07T15:52:28+00:00'
          subject: 'CN=selfsigned.mydomain.co.uk, O=My Org, C=CH'
          thumbprint: 'A5459F3E2FAEA09CFF56DF6BB86E7092CEB4B412'
        }
        encodedCertificate:rootCertificateEncoded
        certificatePassword:rootCertificateEncodedPwd
      }
    ]
  }
}

JamesTran-MSFT 36,366 Reputation points Microsoft Employee

2023-03-16T23:04:55.37+00:00

@Declan O'Malley

Thank you for your detailed post, update, and I apologize for the delayed response!

From your issue, I understand that your APIM connects to an Application Gateway over HTTPS using a self-signed certificate. When uploading this as a certificate (selfsigned.mydomain.co.uk) to the Key Vault and selecting the secret through your Bicep template, APIM is running into an SSL/TLS issue when connecting to your Application Gateway.

However, if you manually upload the Certificate or if you upload the Certificate as a Secret (by base64 encoding) instead of a Certificate within the Key Vault everything works as expected.

To gain a better understanding of your issue, can you share any documentation you followed and the SSL/TLS error message that you ran into? I've also reached out to our APIM team to see if they can look into this issue as well.

If you have any other questions, please let me know. Thank you for your time and patience throughout this issue.
JamesTran-MSFT 36,366 Reputation points Microsoft Employee

2023-03-20T18:31:43.2166667+00:00

@Declan O'Malley

I wanted to check in and see if you had any other questions or if you had a chance to review my previous comment?
JamesTran-MSFT 36,366 Reputation points Microsoft Employee

2023-03-22T21:42:32.5166667+00:00

@Declan O'Malley

I just wanted to check in and see if you had any other questions or if you were able to resolve this issue?
Declan O Malley 1 Reputation point

2023-03-29T08:53:38.09+00:00

Hi, sorry I've not been getting any notifications of updates to this.

The error is something similar to "Failed SSL Negotiation" .

the doc I used was https://learn.microsoft.com/en-us/azure/api-management/api-management-howto-mutual-certificates but then I had to translate that to ARM/BICEP.

but the issue in summary is that a certificate without a password does not work
Declan O Malley 1 Reputation point

2023-03-29T08:55:48.6333333+00:00

Hi, sorry i've not been getting notifications on this issue.

The error was something similar to "Failed SSL Negotiation" .

The article i used to manually configure is https://learn.microsoft.com/en-us/azure/api-management/api-management-howto-mutual-certificates but then i had to translate that manual process to ARM/BICEP

in summary the certificate does not work if its not password but does if it is.

Thanks
JamesTran-MSFT 36,366 Reputation points Microsoft Employee

2023-04-10T19:43:41.38+00:00
@Declan O'Malley

Thank you for following up on this and I apologize for the delayed response!

From your initial post/follow-up it sounds like the certificate doesn't work if it's uploaded as a Certificate, but if you upload the Certificate as a Secret (similar to a password) everything works as expected.

From the documentation you shared, after adding your Certificate to the Key Vault, are you also uploading the Certificate with the password to your API Management instance?

If you're still having problems and would like to work with our Azure Support Engineers, so they can take a closer look into your environment and help resolve your issue. Can you please email me with the info below, I'll go ahead and enable a one-time free technical support request for your subscription so you can work with our engineers to get this issue resolved.

Email: AzCommunity@microsoft.com Subject: ATTN - James Tran Body: Azure Subscription ID Link to this issue

If you have any other questions, please let me know. Thank you for your time and patience throughout this issue.