Partilhar via

DFSR Does Not Replicate Temporary Files

  • Artigo

Note that this post has been added to the TechNet Wiki to allow for community editing.

If you notice that DFS Replication (DFSR) is not replicating certain files, one simple reason is that the temporary attribute is set on them. By design, DFSR does not replicate files if they have the temporary attribute set on them, and it cannot be configured to do so.

This may not be obvious because nearly all the normal methods you would use in Windows to check file attributes do not show the temporary attribute. Specifically, all of the following do not show the temporary attribute - Attrib.exe, Explorer's file properties, FileSystemObject in Windows Scripting Host, and CIM_Datafile in WMI. Also, DFSR does not log any errors to the event log or to the debug logs to show temporary files are not being replicated. There is a relevant entry in the debug logs, but it is not an error because this behavior is by design.

The reason DFSR does not replicate files with the temporary attribute set is that they are considered short-lived files that you would never actually want to replicate. Using the temporary attribute on a file keeps that file in memory and saves on disk I/O. Therefore applications can use it on short-lived files to improve performance.

An application can use FILE_ATTRIBUTE_TEMPORARY when calling the CreateFile function if they want a temporary file. But an even better way is to also specify FILE_FLAG_DELETE_ON_CLOSE so the temporary file is deleted when all handles are closed. That way you get the performance benefit of a temporary file (it’s kept in memory) and it is removed when handles are closed so administrators don’t come along and wonder why DFSR isn’t replicating it.

If you have temporary files that you want DFSR to replicate, you may think it is enough to just remove the temporary attribute on those files and be on your way. And you can do that. But since you got in this situation once, it is likely you still have an application that will come right back and create more temporary files. So you need to get at the crux of the issue – why do you want to replicate files that an application is specifically creating to be temporary? Either the application must change its behavior, or you must except that temporary files won’t be replicated, because there is no way to make DFSR replicate files as long as the temporary attribute is set on them.

Checking the Temporary Attribute on a File using Fsutil

But wait, you say, maybe I don’t even know yet if these files that aren’t replicating are temporary! So let’s find out. As mentioned before, almost none of the ways to check attributes in Windows will actually show the temporary attribute. But there is one that does – the handy Fsutil tool that is included in Windows.

fsutil usn readdata c:datatest.txt

Major Version : 0x2
Minor Version : 0x0
FileRef# : 0x0021000000002350
Parent FileRef# : 0x0003000000005f5e
Usn : 0x000000004d431000
Time Stamp : 0x0000000000000000 12:00:00 AM 1/1/1601
Reason : 0x0
Source Info : 0x0
Security Id : 0x5fb
File Attributes : 0x120
File Name Length : 0x10
File Name Offset : 0x3c
FileName : test.txt

File Attributes is a bitmask that indicates which attributes are set. In the above example, 0x120 indicates the temporary attribute is set because that is 0x100 and 0x20 (Archive) = 0x120.

Here are the possible values:

READONLY 0x1
HIDDEN 0x2
SYSTEM 0x4
DIRECTORY 0x10
ARCHIVE 0x20
DEVICE 0x40
NORMAL 0x80
TEMPORARY 0x100
SPARSE_FILE 0x200
REPARSE_POINT 0x400
COMPRESSED 0x800
OFFLINE 0x1000
NOT_CONTENT_INDEXED 0x2000
ENCRYPTED 0x4000

You combine the values to come up with the File Attributes bitmask value.

If you need a sanity check:

  1. Start, Run, Calc.
  2. Change to Hex and paste in the File Attributes value from the Fsutil command. Say for example, 4925.
  3. Hit the And button, then type 100.
  4. Hit equals and if it returns 100, then the temporary attribute is set. If it returns 0, the temporary attribute is not set.

Checking for Temporary Files in the Debug Logs with Findstr

Another way to check if files are not replicating because they have the temporary attribute set is to use Findstr (included in Windows) to look for the FILE_ATTRIBUTE_TEMPORARY text string in the DFSR debug logs.

First you need to extract out all of the debug logs, because all except the active log will be compressed, as indicated by a .GZ extension. The DFSR debug logs (Dfsr*.log and Dfsr*.log.gz) reside by default under %windir%debug. All the popular compression tools such as Winzip and Winrar can handle .GZ compression.

Let’s say you extracted the debug logs to C:Logs. You can then run the following Findstr command to look for temporary files.

Findstr FILE_ATTRIBUTE_TEMPORARY c:logsdfsr*.log

That will output the entire line for every line in the debug log that contains a match to that string. If it doesn't find any matches, it will return to a prompt and not show anything.

Sample output from a matching entry:

C:WINDOWSdebugDfsr00018.log:20080903 16:14:29.390 1808 USNC 1204 UsnConsumer::ProcessUsnRecord Skipping USN_RECORD with FILE_ATTRIBUTE_TEMPORARY flag:

If it does find any matches, you can then open the specified log file, search on the string FILE_ATTRIBUTE_TEMPORARY (Ctrl+F or Edit | Find in Notepad) and then you will see the actual file name for the file that was skipped because the temporary attribute is set on it.

Removing the Temporary Attribute from Multiple Files with Powershell

So you figured out that DFSR is not replicating some files because they have the temporary attribute set. There is no way to change this behavior in DFSR, so the only option is to live with it, or remove the temporary attribute from the files you want to replicate. An application in your environment has created these temporary files, so just treating the symptom isn’t enough, you need to find the application that creates them and either change its behavior, or accept that those files will not be replicated.

Since Attrib is not aware of the temporary attribute, we need to go to greater lengths to remove it. First you need to have Powershell installed on the machine - www.microsoft.com/powershell

Then bring up a Powershell prompt (Start, Run, Powershell or from the Programs menu) and run this command to remove the temporary attribute from all files in the specified directory, including subdirectories (in this example, D:Data):

Get-childitem D:Data -recurse | ForEach-Object -process {if (($_.attributes -band 0x100) -eq 0x100) {$_.attributes = ($_.attributes -band 0xFEFF)}}

If you don’t want it to work against subdirectories just remove the -recurse parameter.

Additional Information

Win32 CreateFile Function
https://msdn.microsoft.com/en-us/library/aa363858(VS.85).aspx

It’s Only Temporary
https://blogs.msdn.com/larryosterman/archive/2004/04/19/116084.aspx

103237 INFO: Using Temporary Files Can Improve Application Performance
https://support.microsoft.com/kb/103237

131324 Use New Flags to Speed Up C Run-time Low-Level I/O Functions
https://support.microsoft.com/kb/131324

Designing Distributed File Systems (see Files Not Replicated under DFSR Replication Compatibility)
https://technet.microsoft.com/en-us/library/cc772778.aspx

- Craig Landis

Comments

  • Anonymous
    November 11, 2008
    PingBack from http://www.safewordfordforcitrix.com/ask-the-directory-services-team-dfsr-does-not-replicate

  • Anonymous
    February 11, 2009
    Thanks for this very helpful article.  For some reason our file server has ~100 non-temporary files that have this temporary flag set. To find them all, to check prior to updating the flags, I modified your powershell script to: Get-childitem D:Data -recurse | ForEach-Object -process {if(($.attributes -band 0x100) -eq 0x100) {write-output $}}

    • Anonymous
      March 08, 2016
      The comment has been removed

  • Anonymous
    June 11, 2009
    The comment has been removed

  • Anonymous
    June 11, 2009
    You will likely need to examine the debug logs to see why. On a server where one of those files won't replicate out, examine the dfsr debug log in c:windowsdebug for one of those files and paste in what it says for attributes or errors regardin that file. I can take a quick look.

  • Anonymous
    June 11, 2009
    Hey Ned!  I've attached the information in the latest log file that I have related to one of the many files that aren’t copying.  Without posting TON's of information... I've selected what I thought may be of help to you in the log.  Thanks a bunch Ned! Here is the information about one of the files:

  • present                         1
  • nameConflict                    0
  • attributes                      0x2020
  • ghostedHeader                   0
  • data                            0
  • gvsn                            {BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5736678
  • uid                             {BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524452
  • parent                          {BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524410
  • fence                           16010101 00:00:00.000
  • clockDecrementedInDirtyShutdown 0
  • clock                           20090610 15:49:50.494 GMT (0x1c9e9e316ba87bb)
  • createTime                      20080703 13:28:14.290 GMT
  • csId                            {107C5A9A-3267-4DFA-9CFE-7E9834BD3A80}
  • hash                            00000000-00000000-00000000-00000000
  • similarity                      00000000-00000000-00000000-00000000
  • name                            Spooner Invoice 08-1638-1--LINDA LANE----DOTTIE LYNN-- FORTWORTH DANNENBAUM.pdf Here is some information in the same log file that may be a problem: 20090611 08:28:57.689 4172 MEET  4092 Meet::GetParent -> BLOCKED Could not find parent. parentUid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524410 updateName:4278-32 Invoice 05.pdf uid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524450 gvsn:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5736676 connId:{D9434105-3BF1-4C8F-AD54-4E0D5C0511C2} csName:FTWDATA csId:{107C5A9A-3267-4DFA-9CFE-7E9834BD3A80} 20090611 08:28:57.689 2748 MEET  1207 Meet::Install Retries:0 updateName:Spooner Invoice 08-1638-1--LINDA LANE----DOTTIE LYNN-- FORTWORTH DANNENBAUM.pdf uid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524452 gvsn:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5736678 connId:{D9434105-3BF1-4C8F-AD54-4E0D5C0511C2} csName:FTWDATA updateType:remote
  • Anonymous
    June 11, 2009
    Hey Ned!  I've attached the information in the latest log file that I have related to one of the many files that aren’t copying.  Without posting TON's of information... I've selected what I thought may be of help to you in the log.  Thanks a bunch Ned!
  • [Error:9027(0x2343) Meet::DownloadSelfInjectedUpdate meet.cpp:6055 4304 C A failure was reported by the remote partner]
  • [Error:9027(0x2343) Meet::Download meet.cpp:2052 4304 C A failure was reported by the remote partner]
  • [Error:9027(0x2343) InConnection::RdcGet inconnection.cpp:5751 4304 C A failure was reported by the remote partner]
  • [Error:9027(0x2343) DownstreamTransport::RdcGet downstreamtransport.cpp:5201 4304 C A failure was reported by the remote partner]
  • [Error:9027(0x2343) RpcFinalizeContext downstreamtransport.cpp:1096 4304 C A failure was reported by the remote partner]
  • [Error:9027(0x2343) DownstreamTransport::RdcGet downstreamtransport.cpp:5124 4304 C A failure was reported by the remote partner]
  • [Error:9024(0x2340) DownstreamTransport::RdcGet downstreamtransport.cpp:5124 4304 C The file meta data is not synchronized with the file system] 20090611 08:29:05.724 2748 MEET  1207 Meet::Install Retries:0 updateName:4278-32 Invoice 03.pdf uid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524448 gvsn:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5736674 connId:{D9434105-3BF1-4C8F-AD54-4E0D5C0511C2} csName:FTWDATA updateType:remote 20090611 08:29:05.724 5956 MEET  1207 Meet::Install Retries:0 updateName:4278-32 Invoice 05.pdf uid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524450 gvsn:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5736676 connId:{D9434105-3BF1-4C8F-AD54-4E0D5C0511C2} csName:FTWDATA updateType:remote 20090611 08:29:05.724 4564 MEET  1207 Meet::Install Retries:0 updateName:4278-32 Invoice 07.pdf uid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524451 gvsn:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5736677 connId:{D9434105-3BF1-4C8F-AD54-4E0D5C0511C2} csName:FTWDATA updateType:remote 20090611 08:29:05.724 4304 MEET  1207 Meet::Install Retries:0 updateName:Spooner Invoice 08-1638-1--LINDA LANE----DOTTIE LYNN-- FORTWORTH DANNENBAUM.pdf uid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524452 gvsn:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5736678 connId:{D9434105-3BF1-4C8F-AD54-4E0D5C0511C2} csName:FTWDATA updateType:remote 20090611 08:29:05.724 2748 MEET  4092 Meet::GetParent -> BLOCKED Could not find parent. parentUid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524410 updateName:4278-32 Invoice 03.pdf uid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524448 gvsn:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5736674 connId:{D9434105-3BF1-4C8F-AD54-4E0D5C0511C2} csName:FTWDATA csId:{107C5A9A-3267-4DFA-9CFE-7E9834BD3A80} 20090611 08:29:05.724 5956 MEET  4092 Meet::GetParent -> BLOCKED Could not find parent. parentUid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524410 updateName:4278-32 Invoice 05.pdf uid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524450 gvsn:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5736676 connId:{D9434105-3BF1-4C8F-AD54-4E0D5C0511C2} csName:FTWDATA csId:{107C5A9A-3267-4DFA-9CFE-7E9834BD3A80} 20090611 08:29:05.724 4304 MEET  4092 Meet::GetParent -> BLOCKED Could not find parent. parentUid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524410 updateName:Spooner Invoice 08-1638-1--LINDA LANE----DOTTIE LYNN-- FORTWORTH DANNENBAUM.pdf uid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524452 gvsn:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5736678 connId:{D9434105-3BF1-4C8F-AD54-4E0D5C0511C2} csName:FTWDATA csId:{107C5A9A-3267-4DFA-9CFE-7E9834BD3A80} File info from same Log file-
  • present                         1
  • nameConflict                    0
  • attributes                      0x2020
  • ghostedHeader                   0
  • data                            0
  • gvsn                            {BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5736678
  • uid                             {BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524452
  • parent                          {BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5524410
  • fence                           16010101 00:00:00.000
  • clockDecrementedInDirtyShutdown 0
  • clock                           20090610 15:49:50.494 GMT (0x1c9e9e316ba87bb)
  • createTime                      20080703 13:28:14.290 GMT
  • csId                            {107C5A9A-3267-4DFA-9CFE-7E9834BD3A80}
  • hash                            00000000-00000000-00000000-00000000
  • similarity                      00000000-00000000-00000000-00000000
  • name                            Spooner Invoice 08-1638-1--LINDA LANE----DOTTIE LYNN-- FORTWORTH DANNENBAUM.pdf
  • Anonymous
    June 11, 2009
    You have a database issue, where we are missing a PDNT for a folder. You can try the following:
  1. Figure out the parentUID record by searching the debug logs for the GUID and version number referenced in the BLOCKED entry. For example: 20080208 14:29:20.519 804 MEET 3108 Meet::GetParent -> BLOCKED Could not find parent. parentUid:{2A623A3B-43C2-461E-AB2A-ABD99BE38362}-v11496456 {2A623A3B-43C2-461E-AB2A-ABD99BE38362}-v11496456 <- our bad guy
  2. Search the downstream logs for the UID above with entries like: 20080205 11:52:26.123 4952 MEET 4098 Meet::FindNameRelated -> WAIT Name-related record with fid:0x3000000000CB0 has not been processed by UsnConsumer updateName:Scripts uid:{2A623A3B-43C2-461E-AB2A-ABD99BE38362}-v11496456 gvsn:{2A623A3B-43C2-461E-AB2A-ABD99BE38362}-v11496456 connId:{23FEF6CD-7897-4178-90ED-689A8DD2D75D} csName:MTWS csId:{A2BEC2EF-5E74-4A65-978E-3115F93CBA4E}updateName:Scripts <- the folder that has issues in database csName:MTWS <- the repliction group it belongs to
  3. Now we know the folder with an issue.
  4. In the replicated folder, locate this folder (in this case it was SCRIPTS).
  5. Set a trivial ACE on the file (like add Power Users READ) then remove the ACE. All we are doing here is forcing a USN update to the file that cause the correct records to be re-written into the local DFSR database.
  6. Restart the DFSR service on the affected machine - replication should start flowing without issues. If this doesn't work, let me know.

  • Anonymous
    June 11, 2009
    Do I alter the permissions on the upstream or downstream server?  Same question for restarting dfsr.  Also, isn’t there a command that you can run that will give you a list of the backlog files and their path?  The dfsridag backlog only shows the names of the files.  Thanks!

  • Anonymous
    June 11, 2009

  1. To ensure that i understand this correctly i've done this-  On the updstream server i searched for "BLOCKED" and copied the UID of the problem file's parent- 20090611 08:23:47.187 2748 MEET  4092 Meet::GetParent -> BLOCKED Could not find parent. parentUid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5730210 updateName:ned3297.dem uid:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5730229 gvsn:{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5736815 connId:{D9434105-3BF1-4C8F-AD54-4E0D5C0511C2} csName:FTWDATA csId:{107C5A9A-3267-4DFA-9CFE-7E9834BD3A80} Problem ParentUID- :{BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5730210.
  2. On the downstream server i searched the logs and from the latest log file found the UID and found the following information but was unable to find on any line pretaining to that UID the "Updatename" section.  I did find that it appreas that the DFSR database doesnt know the name of the folder and instead is substituting the UID.  See below-
  • present                         1
  • nameConflict                    0
  • attributes                      0x0
  • ghostedHeader                   0
  • data                            0
  • gvsn                            {00000000-0000-0000-0000-000000000000}-v0
  • uid                             {BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5730210
  • parent                          {00000000-0000-0000-0000-000000000000}-v0
  • fence                           16010101 00:00:00.000
  • clockDecrementedInDirtyShutdown 0
  • clock                           16010101 00:00:00.000 GMT (0x0)
  • createTime                      16010101 00:00:00.000 GMT
  • csId                            {107C5A9A-3267-4DFA-9CFE-7E9834BD3A80}
  • hash                            00000000-00000000-00000000-00000000
  • similarity                      00000000-00000000-00000000-00000000
  • name                            {BEDA9BFB-254E-4A8A-B3C6-168D8E5A0034}-v5730210 Thanks in advance!!

  • Anonymous
    June 11, 2009
    Upstream for both. You cannot see backlog paths until you get to 2008 R2 using DFSRDIAG, and even then it will be not a full backlog but instead the current queue.

  • Anonymous
    January 13, 2010
    For what it is worth to someone who may stumble upon this,  I ran across a problem where pdf files were not being replicated by the DFSR engine because they had been tagged as temporary files. I used treesize professional to find out that it was PDF files not being replicated and used the same program to search for all temporary files.  I was able to narrow this down to one user who was using PDF995, a free pdf printer, to convert documents to pdfs.  There is something about this program that creates a PDF file that is tagged as temporary.  They were using the trial version and it was several years old.  I do not know if the new version still does this.  I hope this may help somebody.  

  • Anonymous
    April 15, 2014
    Hello, this is Atanu Dey here to talk about the known issues with DFS Replication. We are observing that