What Causes "The payload was invalid" Error in .Net Core 3.1 Application?

Ty Nguyen 1

We have a .Net Core 3.1 web app that uses Microsoft.AspNetCore.DataProtection version 3.1.0 to encrypt and decrypt data. The application all of the sudden fails to decrypt the data because of the error "The payload was invalid" as seen below:

[2021-08-18 08:12:19 ERR] [FoxCentral.Web.Api.ErrorController] Path: /api/botflows/2. Error: The payload was invalid.
Trace: at Microsoft.AspNetCore.DataProtection.Cng.CbcAuthenticatedEncryptor.DecryptImpl(Byte* pbCiphertext, UInt32 cbCiphertext, Byte* pbAdditionalAuthenticatedData, UInt32 cbAdditionalAuthenticatedData)
at Microsoft.AspNetCore.DataProtection.Cng.Internal.CngAuthenticatedEncryptorBase.Decrypt(ArraySegment1 ciphertext, ArraySegment1 additionalAuthenticatedData)
at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.UnprotectCore(Byte[] protectedData, Boolean allowOperationsOnRevokedKeys, UnprotectStatus& status)
at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.DangerousUnprotect(Byte[] protectedData, Boolean ignoreRevocationErrors, Boolean& requiresMigration, Boolean& wasRevoked)
at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Unprotect(Byte[] protectedData)

We store the keys in a database using Entity Framework Core and use X509 certificates to protect the keys. Below is how we set up data protection in our app:

var protectionBuilder = services.AddDataProtection();

protectionBuilder.PersistKeysToDbContext<KeysContext>();

protectionBuilder.ProtectKeysWithCertificate(certificates.KeyProtectCertificate)
.UnprotectKeysWithAnyCertificate(certificates.KeyUnprotectCertificates.ToArray());

All the data was encrypted and decrypted on the same server. What causes that decryption failure? How to recover the data?

Chao Deng-MSFT 796 Reputation points

2021-08-19T08:00:26.653+00:00

Hi @Ty Nguyen ,
You encrypt and decrypt the data on the same server and store the key in the database, but by default, a new key will be regenerated every 90 days, so that if the ciphertext in the database is not updated, it will Failure, whether this is the reason, If not, can you provide more relevant configuration information?

In addition, the official documentation may also be helpful to you.

https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/configuration/overview?view=aspnetcore-5.0
Ty Nguyen 1 Reputation point

2021-08-20T16:23:07.063+00:00

Do we have to manually update the ciphertext in the keys database? I thought that would be handled by the data protection API automatically. What causes the ciphertext update to fail? The biggest question is how to recover the data?
Chao Deng-MSFT 796 Reputation points

2021-08-24T02:22:45.1+00:00

Hi @Ty Nguyen ,
When a key expires, the app automatically generates a new key and sets the new key as the active key. As long as retired keys remain on the system, your app can decrypt any data protected with them. The data protection system manages key rotation internally and creates a new key when the old key expires.See key management for more information.
In addition,expired keys can be used to decrypt existing data, but they can't be used to encrypt new data. If you're running your application in a clustered scenario, you'll want to take a look at one of the alternative configuration approaches.
Dean Jackson 1 Reputation point

2022-09-24T15:08:39.31+00:00

@Chao Deng-MSFT
If the key is expired but still exists, and we get the "payload was invalid" error when unprotecting data that was encrypted with that key, is that a bug in .NET? Seems like it is.
I'm new to this and wanted to run a test to see how it works. I encrypted some simple text and saved the text to a file. Then waited for my key to be expired. Now that it's expired, I see a new key has been created, but the old key still exists at: C:\Users\MyName\AppData\Local\ASP.NET\DataProtection-Keys

However, when I try to unprotect the test string, I get the "payload was invalid" error. Do I have to do anything to specify the old key?
Thanks.
Dean Jackson 1 Reputation point

2022-09-24T15:28:02.923+00:00

I think my problem was not remembering either the ApplicationName (ApplicationDiscriminator") OR the purpose value, because if you don't have either of these correct, then you will get this "payload was invalid" error.

Share via

What Causes "The payload was invalid" Error in .Net Core 3.1 Application?