You can try pressing CTRL+shift+ESC to open the task manager, find "Windows Explorer" and select restart.
Then try to delete again.
If that doesn't remove it, you can remove the problem with a clean boot.
The method of clean boot can refer to this:
How to perform a clean boot in Windows (microsoft.com)
Disclaimer: A “clean boot” starts Windows with a minimal set of drivers and startup programs. It helps to determine whether a background program is interfering with your game or program and to isolate the cause of a problem.
These steps of "clean boot" might look complicated at first glance. However, to avoid any trouble for you, please follow them in order and step-by step so that it will help you get you back on track.