A family of Microsoft word processing software products for creating web, email, and print documents.
As Suzanne says, the temp files should be deleted automatically when the document is closed (not when it is saved). Various things can interfere with that, including Word program crashes, poorly written add-ins, and improperly set folder permissions. Without going through a tedious troubleshooting process, there's no way to know why your temp files stay.
What you can do is set up a batch file that deletes the files when you run it. A sample batch file -- with important notes about what you must change to make a working file from the sample -- is in http://www.gmayor.com/what_to_do_when_word_crashes.htm .