Hello @Mads Westen ,
Thanks for reaching out.
Extending the expiration date is not supported, but as you mentioned you can create new credential to use a custom value instead by using the following cmdlet, which allows you to set prior PasswordCredentials granted to the application.
New-AzureADApplicationPasswordCredential -ObjectId 34c04e6c-a0e1-46be-b0c2-14f1fb052391 -Value "Nez6kRbXgtphaIU0kQy2Jnn5SCj1CVMAV1xTYL8ABEU=" -CustomKeyIdentifier Test
Hope this helpful.
-----
Please "Accept the answer" if the information helped you. This will help us and others in the community as well.