I wonder if there is something to do with a history of attachments having been in the database as I had experienced old attachments reappearing when creating new table records - but hadn't fathomed out why!).
I would suspect this is a clue to the issue. Deleting records should also delete the attachment records, and if it's not doing that then you would likely have issue with either (a) the database or (b) the environment.
If it's the database, then most times your C&R would fix the issue. You may also try a Decompile to see if that works. To do that, create a standard Shortcut with this Target:
"full path to msaccess.exe" /decompile
Run that shortcut, and then open your database from within that session of Access. After opening, compact your database again, and compile your code (click Debug - Compile from the VBA Editor, and fix any errors you find).
If it's environmental, you should ensure that your installation of Office and Windows is fully up to date.