Usually deleting a web part also requires deleting from the site collection web part gallery. Edit the section to be removed in Manage Web Part Gallery under Site Settings, click on the menu item "Delete".
But as far as I know, if there are references to custom web parts in Site Features and Site Collection Features, they need to be removed through Power Shell. You can try the commands in this article, but it's a bit old and not sure if it will work with newer versions of SP.
stsadm -o retractsolution -name *wspfile file name* -url site url -immediate
If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.