@TKentrup For any replying party SSO certificate, you need to make the changes at the following location :
1) Login to Azure Portal
2) Go to Azure Active Directory
3) Enterprise Applications
4) Search your application name
5) Once into the Application properties, select Single Sign On :
6) Edit the Certificate part
If the suggested response helped you resolve your issue, please do not forget to accept the response as Answer and "Up-Vote" for the answer that helped you for benefit of the community.