mutual TLS fails when using ECDSA

Question

Hi,

I have an Azure Web App configured to accept client certificates for authentication (mTLS). Yesterday clients started failing to connect to the service with SSL "bad record mac" error. The requests are failing before they reach my endpoint. The certificates I use are ECDSA signed. When testing with RSA signed certificate, the request reaches my endpoint.

Note: This has affected all my Azure Web Apps and Functions at the same time

Below is the error message I get when I call my endpoint. What should I do?

myuser@server:$ curl -v --cert client.pem --key client-key.pem https://yyyyy.azurewebsites.net/register/456456

*   Trying xx.xx.xx.xx:443...

* TCP_NODELAY set

* Connected to yyyyy.azurewebsites.net (xx.xx.xx.xx) port 443 (#0)

* ALPN, offering h2

* ALPN, offering http/1.1

* successfully set certificate verify locations:

*   CAfile: /etc/ssl/certs/ca-certificates.crt

  CApath: /etc/ssl/certs

* TLSv1.3 (OUT), TLS handshake, Client hello (1):

* TLSv1.3 (IN), TLS handshake, Server hello (2):

* TLSv1.2 (IN), TLS handshake, Certificate (11):

* TLSv1.2 (IN), TLS handshake, Server key exchange (12):

* TLSv1.2 (IN), TLS handshake, Server finished (14):

* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):

* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):

* TLSv1.2 (OUT), TLS handshake, Finished (20):

* TLSv1.2 (IN), TLS handshake, Finished (20):

* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384

* ALPN, server accepted to use http/1.1

* Server certificate:

*  subject: C=US; ST=WA; L=Redmond; O=Microsoft Corporation; CN=*.azurewebsites.net

*  start date: Mar 10 03:05:55 2023 GMT

*  expire date: Mar  4 03:05:55 2024 GMT

*  subjectAltName: host "yyyyy.azurewebsites.net" matched cert's "*.azurewebsites.net"

*  issuer: C=US; O=Microsoft Corporation; CN=Microsoft Azure TLS Issuing CA 02

*  SSL certificate verify ok.

> GET /register/456456 HTTP/1.1

> Host: yyyyy.azurewebsites.net

> User-Agent: curl/7.68.0

> Accept: */*

>

* TLSv1.2 (IN), TLS handshake, Hello request (0):

* TLSv1.2 (OUT), TLS handshake, Client hello (1):

* TLSv1.2 (IN), TLS handshake, Server hello (2):

* TLSv1.2 (IN), TLS handshake, Certificate (11):

* TLSv1.2 (IN), TLS handshake, Server key exchange (12):

* TLSv1.2 (IN), TLS handshake, Request CERT (13):

* TLSv1.2 (IN), TLS handshake, Server finished (14):

* TLSv1.2 (OUT), TLS handshake, Certificate (11):

* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):

* TLSv1.2 (OUT), TLS handshake, CERT verify (15):

* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):

* TLSv1.2 (OUT), TLS handshake, Finished (20):

* TLSv1.2 (OUT), TLS alert, bad record mac (532):

* OpenSSL SSL_read: error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac, errno 0

* Closing connection 0

curl: (56) OpenSSL SSL_read: error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac, errno 0

Answer 1

We checked internally and a quick search does not appear to report anyone else having a similar issue, which leads me to believe the best area to start is reviewing your end.
It seems that the issue is related to the ECDSA signed certificates. The error message "bad record mac" indicates that the SSL/TLS connection is failing due to a decryption error or a bad record message authentication code (MAC).

To resolve this issue, you can try the following steps:

Check if the client certificates are valid and not expired. You can use OpenSSL to check the certificate validity and expiration date.

Check if the client certificates are signed by a trusted certificate authority (CA). You can use OpenSSL to check the certificate chain and verify if the CA is trusted.

Check if the Azure Web App is configured to use the correct TLS version and cipher suite. You can use the Azure Portal or Azure CLI to configure the TLS version and cipher suite.

1. Check if the Azure Web App is configured to use the correct SSL/TLS binding. You can use the Azure Portal or Azure CLI to configure the SSL/TLS binding.

If you are still having issues even after following the above steps, please reply back here so we can assist you further.

Answer 2

Hi again @brtrach-MSFT

Thanks for your answer, please see my reply to your steps in addition to some more info below:

Check if the client certificates are valid and not expired. You can use OpenSSL to check the certificate validity and expiration date.

Certificates are checked to not be expired

Check if the client certificates are signed by a trusted certificate authority (CA). You can use OpenSSL to check the certificate chain and verify if the CA is trusted.

The CA certificate is self signed. I don't see how that would affect mutual authentication though. The client certificate is not reaching the endpoint for verification against the CA.

Check if the Azure Web App is configured to use the correct TLS version and cipher suite. You can use the Azure Portal or Azure CLI to configure the TLS version and cipher suite.

What configureation items are you referring to? What is the correct version?

Check if the Azure Web App is configured to use the correct SSL/TLS binding. You can use the Azure Portal or Azure CLI to configure the SSL/TLS binding.

Where do I find the binding configuration? What is the correct binding?

I have done some more testing:
I have used OpenSSL to generate EC keys on the following curves secp224r1, secp256k1, secp384r1, secp521r1, and prime256v1. Then used each to sign a certificate and submit a request to the API. The reported error occurs with all the curves except secp384r1 and prime256v1.
Openssl version "OpenSSL 3.0.8 7 Feb 2023 (Library: OpenSSL 3.0.8 7 Feb 2023)"