Outlook uses a file "OUTLOOK.HOL" which is a text file you can edit. It contains the holidays for every country.
In the "United States" section, The file I have does not include Thanksgiving beyond the year 2022. So no Thanksgiving this year for me.
When you "add holidays" you are simply asking Outlook to open this file and add the holidays in it. If you do it twice, the holidays will be created twice, etc. So, you may just want to find a current ".HOL" file or edit the one you have.
See the following for more info.
https://www.outlook-tips.net/how-to/missing-holidays/
https://www.slipstick.com/outlook/calendar/outlooks-holiday-file/