What is the best way to detect a change in a file and make a backup of it when it changes?

Cynolycus 255

As the title says I want ideas on the best way to detect a change in a file and create a backup of it when it changes.

I have an external program that saves or updates a file at irregular intervals.

I want my app to detect any change in this file and create a backup of it.

Several years ago I created an app that did this using FileSystemWatcher and it saved the backups with the time included in the name.

The problem that I had with it was that the external program seemed to update the file in pieces, so every time it saved the file it was actually saving it multiple times and only the last time it saved was it complete.

If I remember correctly I built in a timer so my app would make a backup as soon as the file changed then it would wait a short time and make another backup if there was another change detected during that time, and if no change was detected it didn't create another backup.

This meant that the number of backups was reduced to two or three backups each time the file was saved, but only the last backup was complete.

I have created another app that does much more but backing up this file is part of it.

Should I do it the same way as before but maybe have it keep track of the backups it makes each time and delete all but the last backup, or is there a better way to do this?

Anonymous

If you check various sites for "FileWatcher", you will see that the issue you are seeing is in fact a bug in the FileWatcher. Any general search will point it out.

To avoid that, I used a different approach. Here is an stand alone example. If you want to try it out, you will need a new test Project with a Form1, RichTextBox named 'RTB', a Timer1 and a My.Settings item named 'lwt'. The code is simplified and has no exception handling.

With limited testing, this example works fine here (the long term code I used has never had issues). I would give a passing thought to handling any exception which could be caused by the backup code trying to access the file at the same time your remote application was writing to it.

' Form1 with Timer1,
' RichTextBox (here named RTB)
' My.Settings.lwt as long
Option Strict On
Option Explicit On
Public Class Form1

  ' set up folder/paths structure

  Dim FolderToCheck As String = IO.Path.Combine(My.Computer.FileSystem.SpecialDirectories.Desktop, "TestForChanges")

  Dim FileNameToCheck As String = IO.Path.Combine(FolderToCheck, "FileToCheck.txt")

  ' I just put the backup folder inside
  ' same folder as file to backup
  Dim BackUpFolder As String = IO.Path.Combine(FolderToCheck, "BackUpFolder")

  Dim BackUpFilePath As String = IO.Path.Combine(BackUpFolder, IO.Path.GetFileName(FileNameToCheck))
  Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
    With Timer1
      ' fire every 2 sec for test,  a much
      ' longer interval would be more usual
      .Interval = 2000
      .Enabled = True
    End With
    Dim d As Date = New Date(My.Settings.lwt)
    AppLine("Started: Last BackUp was " & New Date(My.Settings.lwt).ToString("ddd, dd MMM yyyy HH:mm:ss"))
  End Sub

  Sub AppLine(s As String)
    RTB.AppendText(s & vbCrLf)
  End Sub

  Private Sub Timer1_Tick(sender As Object, e As EventArgs) Handles Timer1.Tick
    Dim lastbackup As Long = IO.File.GetLastWriteTime(FileNameToCheck).Ticks

    If lastbackup > My.Settings.lwt Then
      ' make new backup. Could TimeStamp
      ' the name here if needed)
      IO.File.Copy(FileNameToCheck, BackUpFilePath, True)
      My.Settings.lwt = lastbackup

      ' just to show event in RTB, would more
      ' likely be in a log file.
      AppLine("New Back Up made: " & New Date(My.Settings.lwt).ToString("ddd, dd MMM yyyy HH:mm:ss"))
    End If
  End Sub
End Class

Cynolycus 255 Reputation points

2023-11-16T00:22:04.5433333+00:00

Thanks Les, after a little more looking and reading I did find a few more articles and posts on the FileWatcher.

After reading them I had considered removing this post but I thought it might be good to see what approach others find best to use.

One person didn't use the FileWatcher at all because they thought it was unreliable, they just used a timer and at each interval they checked the file's last write time. I'm not sure I like this approach.

I did find an article explaining that the FileWatcher's OnChange is fired by more than the last write time, I guess that is what you are referring to as a bug in the FileWatcher.

After reading the article I had two ideas, one was much the same as your approach and the other is a little different and doesn't use a timer.

Instead of using a timer I had thought about storing the last write time in a variable and then each time the FileWatcher's OnChange occurred, compare the stored time with the time currently on the file and if they were different then backup the file and update the time in the variable. This would depend on whether the last write time is changed by any other event other than saving the file.

I don't know if this approach is any better than using a timer, but at least I now have two options to explore.
Anonymous

2023-11-16T13:45:56.8666667+00:00

Hi

The example code I posted, more or less, does exactly what you describe, the only difference is that you suggest using the FileWatcher's OnChange to notify your code and in my code, the Timer Tick event serves that purpose. As for which to use. My vote would be Timer as the FileWatcher's OnChange event is flagged as troublesome at times. Also, though I have not illustrated it, I think exception handling would be more straightforward if using a Timer as there is only one code line that needs to be catered for, and in a worst case, an exception can be thrown away (where appropriate) and let the Timer process the next call where the exception cause is 'gone' so to speak. I am thinking of 'file in use' type exceptions there.
RLWA32 44,951 Reputation points

2023-11-17T06:47:23.3366667+00:00

For some pertinent background information about why FileSystemWatcher change notifications behave as they do I suggest reading these two Microsoft blog articles -

Why does saving a file in Notepad fire multiple FindFirstChangeNotification events

ReadDirectoryChangesW reads directory changes, but what if the directory doesn't change?
TP 89,576 Reputation points

2023-11-17T06:59:40.9066667+00:00

"best" way would be to use USN Change Journal, however, it may not be the best in terms of amount of time/effort as compared to other methods
Cynolycus 255 Reputation points

2023-11-18T14:36:49.82+00:00
I don't know for sure but I thought that the FileSystemWatcher would always be running in Windows and therefore if I used it I would reduce the resources needed for the project.

About a year ago I did read somewhere that timers, if constantly running, can take up a lot of the resources, but that could have been an old article from when computers weren't as good as they are now.

I tried both ways and for some reason, even though I was using Notepad to save the text file, I didn't have the trouble of multiple saves using the FileSystemWatcher in the test project, but the multiple threads gave me a headache when I put the code in my project.

So now I have two versions of my project, one using the timer and the other using the FileSystemWatcher and have yet to test either of them with the external program.

This is what I ended up with for the test using the FileSystemWatcher.

Imports System.IO Public Class Form1 Dim SourceDirectory As String = IO.Path.Combine(My.Computer.FileSystem.SpecialDirectories.Desktop, "TestForChanges") Dim BackupDirectory As String = IO.Path.Combine(SourceDirectory, "BackUpFolder") Public watchfolder As FileSystemWatcher Private Sub btn_startwatch_Click(sender As Object, e As EventArgs) Handles btn_startwatch.Click watchfolder = New System.IO.FileSystemWatcher(SourceDirectory) With { .IncludeSubdirectories = False, .NotifyFilter = NotifyFilters.LastWrite } AddHandler watchfolder.Changed, AddressOf watchfolder_OnChanged watchfolder.Filter = "FileToCheck.txt" watchfolder.EnableRaisingEvents = True End Sub Private Sub watchfolder_OnChanged(ByVal sender As Object, ByVal e As EventArgs) My.Computer.FileSystem.CopyFile(SourceDirectory & "\FileToCheck.txt", BackupDirectory & "\" & Now.ToString("dd.MM.yyyy_HH.mm.ss") & ".txt") End Sub Private Sub btn_stop_Click(sender As Object, e As EventArgs) Handles btn_stop.Click watchfolder.EnableRaisingEvents = False End Sub End Class

In my project when the FileSystemWatcher detects a change in the file I wanted to call a sub that I have to manipulate the directories and file names, but the sub had controls that had trouble with the threads, and there were a lot of controls.

I didn't want to deal with that now, so I bypassed the problem by adding an invisible textbox and when the change in the file was detected I invoked that single textbox, changed it's text and used it's textchanged event to call the sub, then at the end of the sub I cleared the textbox. It's probably not the preferred way but it seems to work fine.

I'll learn about threads another day.

Share via

What is the best way to detect a change in a file and make a backup of it when it changes?

Your answer