Unfortunately, it is a known issue with MS Office files. They create temporary files where they store state between save operations and then they merge them to a new file and delete all previous versions of document. Here is a KB article that explains how temporary files work in MS Office: https://support.microsoft.com/en-us/help/211632/description-of-how-word-creates-temporary-files
Unfortunately, there is no way to redirect these temporary files anywhere else, because the path is hardcoded, i.e. in same folder as original document. This means that you have to allow Delete permissions on parent folder for MS Office files purposes. You are stuck with this.