Hi,
Since Private Endpoint NIC is not associated with a compute resource the behavior might be different.
Can you check if you have any VMs deleted or stopped in the same subnet as that of the PE ?
Even if that happens, that shouldn't be the ideal case, for which we might need to raise a SR to figure out if this is a bug.
Regards,
Karthik Srinivas