You should go to your modern OneDrive site to delete the folder shortcut.
Go to the SharePoint document library -> Click App launcher in the upper left corner -> OneDrive -> My Files -> Delete the folder shortcut.
After you delete the folder shortcut, then sync document library in the file explorer.
If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.