Using robocopy on file server and local server

Question

Hello everyone,

I am new to Robocopy and there are some things that I do not really understand.

I am using robocopy to mirror files from the file server (Windows 2012R2) to a local server (Windows 2016) with the following settings:

/MIR /V /NP /R:10 /W:30

In the past, these scheduled tasks very succesful and ten minutes were enough. Now, I face some issues and it is difficult for me to understand it and its roots. I have the following findings:

1. The log files show that many files are either "tweaked" or "modified" even though they have not been changed. What is checked when the output shows "tweaked"/"modified"? I checked the files in file server and local server and I see that "Created date" is different which makes sense. They existed first in the file server.
2. Some jobs took 20 min. up to 9 hours. What can I check if I experience such a long duration?
3. I have read about the option "/FFT: Assumes FAT file times (2-second granularity) to handle timestamp differences between different file systems." -> I do not reallly understand the use of it. It helps to copy between file systems but is it necessary for two different windows systems? I also read something about EXT and NFTS which confuses me. Is there anyone who has experience with using robocopy for local server and file server? I would highly appreciate any help and also information so I can learn and be more independet! Thanks a lot.

Answer 1

We used to use McAfee as our AV solution. When we were doing a big migration, we would turn it off on the destination server because every file write was getting scanned for viruses. That was not needed because the files had already been scanned on the source server and it slowed down the copy process. The subsequent delta copies (/mir) weren't as bad because it was typically copying much less data. Your results will vary depending on your data and AV software.

Your issue is most likely with some other resource. Cpu, memory, disk, network. Start with task manager and check to see if there is sufficient memory available on both source and destination servers. Then use the Details tab and look to see what processes are using the most cpu. See if the Windows Defender, MsMpEng.exe, or your AV software, is using a lot of cpu.

Look at the disk to see how active it is and look at it's response time. Resource Monitor will show the files being accessed. There should also be a graph for disk queue length. If you see queuing, then the disk cannot keep up with all of the concurrent I/O requests.

You may be experiencing a network delay. That could be caused by other servers/PCs/devices on your network. Your source and destination servers might not have visibilty into that. Again use Task Manager and Resource Monitor to watch network activity.

To test the network throughput between 2 machines, you can use the iperf tool.

https://www.techtarget.com/searchnetworking/tip/How-to-use-iPerf-to-measure-throughput

If you see that your network performance is really bad at certain times, then you will need to work with your network team to figure out if they have a hardware issue on a router/NIC or figure out which devices are monopolizing the network.

There are many sites that document how to use perfmon to analyze performance. That would be your next best step.

https://www.bing.com/search?q=how+to+use+perfmon+to+analyze+performance

Tweaked means that a property like read-only was set on the source.

https://superuser.com/questions/314503/what-does-robocopy-mean-by-tweaked-lonely-and-extra

Robocopy will copy the attributes when it copies the file, but if an attribute is set after that, it won't copy just the attribute.

Here is a Powershell script that will copy the attributes. I have commented out the update statement so that it will not change anything. You should run it and examine the files that it reports on. Verify that the attributes are the issue. You may have something different in your environment that I won't know about.

$src = 'c:\temp\xxxx'
$dst = 'c:\temp\xxxxNew'
$files = Get-ChildItem -Path $src -recurse -file
foreach ($sf in $files) {
    $dfname = $dst +  $sf.FullName.Substring($src.length)
    if (test-path $dfname) { 
        $df = Get-Item $dfname
        if ($sf.Attributes -ne  $df.Attributes) {
            # Uncomment to actually update the attributes 
            # $df.Attributes = $sf.Attributes                     # This will not work for Compressed. 
            "Updated: {0} - {1}" -f $sf.Attributes, $dfname
        } 
    } else {
        "Not found: {0}" -f  $dfname
    }
}