Graph API addPassword - Secret not shown in portal

Question

I was exploring the use of the Graph API in conjunction with a Logic App to regenerate application secrets. I get a successful response back from the API with the new secret information, however, when navigating to the application in the portal I don't see any secrets associated with the SPN. The activity log on the application shows that the SPN had passwords added successfully. Why is this?
This is the API I'm calling https://graph.microsoft.com/v1.0/servicePrincipals(appId='{id}')/addPassword and this is the 200 response I get.

{
  "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#microsoft.graph.passwordCredential",
  "customKeyIdentifier": null,
  "displayName": "testsecret",
  "endDateTime": "2023-07-13T00:00:00Z",
  "hint": "hidden for post",
  "keyId": "hidden for post",
  "secretText": "hidden for post",
  "startDateTime": "2023-04-13T00:00:00Z"
}