A simple way to access Shadow Copies in Vista
In the past, I presented various ways to browse these mysterious device objects called "shadow copies". Shadow copies are static images in time (snapshots) of your volume contents, at some point in the past. These shadow copies are volumes on their own, with a file system namespace accessible through the regular Win32 APIs such as FindFirstFile/FindNextFile. For example the existing sample code in MSDN for these APIs that enumerates files on a real volume will work just fine on a shadow copy volume. In fact, that's how all backup applications are accessing shadow copy content today.
So, if these devices are real volumes, how can we view them in Explorer? It turns out that you can't view them by default - this is simply because these are volumes without an associated drive letter or root mount point. However, in XP or Windows Server (and Vista), you can still access these shadow copies by assigning them a drive letter using utilities like DOSDEV, or by doing tricks with the FOR command, etc.
Now, if you have Vista, it is much simpler to access shadow copy devices directly from Explorer. The trick is to use a new feature called Symbolic Links: to access the contents of a shadow copy as a "directory", simply create a symbolic link to the device. Vista also includes a convenient command-line tool called MKLINK.EXE to create symbolic links, which makes this operation very easy.
Here is an example of accessing the contents of a shadow copy device. The first step is to enumerate shadow copies on the machine, using the VSSADMIN LIST SHADOW command. This will give us the devices and also a creation timestamp.
C:\Windows\system32>vssadmin list shadows |more
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Contents of shadow copy set ID: {c72c8036-d563-43c8-b351-1994dfad580a}
Contained 1 shadow copies at creation time: 2/23/2008 9:59:04 AM
Shadow Copy ID: {f3727808-bea6-4b59-bef7-6849ee721709}
Original Volume: (C:)\\?\Volume{3e83355f-7c0e-11dc-b416-806e6f6e6963}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy4
Originating Machine: Adi-Game-PC
Service Machine: Adi-Game-PC
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessibleWriters
Attributes: Persistent, Client-accessible, No auto release, Differential, Auto recovered
Contents of shadow copy set ID: {0bf23f77-8461-4869-b391-da4d213940a5}
Contained 1 shadow copies at creation time: 2/24/2008 4:00:24 AM
Shadow Copy ID: {87d59b22-9e84-4d0d-81ca-2b565d6f7e55}
Original Volume: (C:)\\?\Volume{3e83355f-7c0e-11dc-b416-806e6f6e6963}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy5
Originating Machine: Adi-Game-PC
Service Machine: Adi-Game-PC
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessibleWriters
Attributes: Persistent, Client-accessible, No auto release, Differential, Auto recovered
C:\Windows\system32>mklink /d c:\shadowcopy \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy5\ symbolic link created for c:\shadowcopy <<===>> \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy5\
C:\Windows\system32>dir c:\shadowcopy
Volume in drive C has no label.
Volume Serial Number is 4A02-860C
Directory of c:\shadowcopy
12/14/2007 01:46 AM <DIR> Asi
01/15/2008 12:56 AM <DIR> bin
12/13/2007 11:59 PM <DIR> debuggers
12/13/2007 11:55 PM 17,644,031 dir.log
01/14/2008 11:41 PM <DIR> Downloads
01/01/2008 05:50 PM <DIR> dumps
12/30/2007 11:43 PM <DIR> garbage
01/08/2008 11:13 PM <DIR> Garmin
10/15/2007 09:03 PM <DIR> Intel
12/30/2007 11:59 PM <DIR> Program Files
01/27/2008 01:32 AM <DIR> Program Files (x86)
01/15/2008 12:17 AM <DIR> test
01/30/2008 06:52 AM <DIR> Users
12/14/2007 01:55 AM <DIR> WinDDK
02/13/2008 05:23 AM <DIR> Windows
02/21/2008 10:43 PM <DIR> Work
1 File(s) 17,644,031 bytes
15 Dir(s) 147,657,666,560 bytes free
That's it. Now I have a persistent link called c:\shadowcopy which points to the contents of the shadow copy device - which is the image of my C:\ drive at 4:00 AM (this is when my latest system restore point was created).
A new notes,though:
1) Make sure you use the "/D" option in MKLINK so you create a directory-based, not a file-based symbolic link
2) Make sure you append a backslash to the shadow copy device in the MKLINK command (marked in red above)
If this made you interested about shadow copies - note that you can create, enumerate and delete shadow copies programatically using either VB scripts that use the WMI API for shadow copy administration, or by using the VSS API (documented publicly on MSDN). Sample code is available in the Platform SDK as well.
Comments
Anonymous
February 28, 2008
Wouldnt this also be possible on win2000+ using NTFS junctions? There are no built in commands for this, but using the apis that would work the same right?Anonymous
February 28, 2008
The comment has been removedAnonymous
February 28, 2008
Isn't the simplest way to access them in Vista simply to right-click the drive and click "Restore Previous Versions" and open the desired shadow copy?Anonymous
February 28, 2008
>>> Isn't the simplest way to access them in Vista simply to right-click the drive and click "Restore Previous Versions" and open the desired shadow copy? Correct - this method would work as well.Anonymous
February 29, 2008
The "Restore previous versions" only works if you have setup an ACTUAL shadow, as opposed to the built in vista system restore points though right?Anonymous
March 03, 2008
No, that's the beauty of it. System restore effectively checkpoints your files too, and you can grab them very easily (although the feature isn't nearly as discoverable as it was in XP with the folder panes). It's one of those features you rarely need, but boy is it helpful when you do need it. Unfortunately Home Basic and Home Premium don't have this feature (stupid IMO). But we can rant about the version idiocy another day (really, why does Home Basic even exist?)Anonymous
March 03, 2008
By the way, another thing is that Previous Versions also works in tandem with the Vista backup tool. Unfortunately said backup tool is such a disgrace that it's essentially unusable, so I'm not sure how well this works. But if you're willing to trust a vague description of what file types will be backed up without being able to explicitly select which files and folders you want, have at it. I think this might work in Home Premium.Anonymous
March 04, 2008
>> No, that's the beauty of it. System restore effectively checkpoints your files too, and you can grab them very easily (although the feature isn't nearly as discoverable as it was in XP with the folder panes). It's one of those features you rarely need, but boy is it helpful when you do need it. Unfortunately Home Basic and Home Premium don't have this feature Correction - note that ALL Vista SKUs have System Restore snapshots enabled by default (including Started edition, Home Basic, Home premium, etc) That said, System Restore is not present on server operating system (Windows Server 2008) as you usually have to rely on specialized server-grade backup tools to backup restore system state.Anonymous
March 04, 2008
Right, I was imprecise; System Restore is present in all Vista SKUs but the "Restore Previous Versions" functionality is not. Unfortunately ntbackup is gone in WS 2008 as well, replaced with something similarly anemic to Vista. The image-based backup (only present in Business, Enterprise, and Ultimate - also present in Server) is nice but not being able to choose what files and folders makes the other thing utterly useless.Anonymous
March 11, 2008
Thanks for the method! However, I've been investigating ShadowCopies in depth for a week or so now and would ask where one can find detailed - really low level stuff, not the API - information on how it works... e.g. Is VSS really creating a 16KB block for every file in the snapshot (even though they may not be updated if the regkey FIlesNotToSnapshot excludes them... which I can sort of understand: file filtering at copy-on-write would undermine the whole principle...)? Why? (if there is little churn it unnecessarily profligate with space) etc. etc. etc. My problem is that on a Vista HP laptop (i.e. without the nominal benefits of Previous Versions in other SKUs) System Restore points are so large there is no useful Restore Point history... I am trying to gain a little control (reduce the frequency of scheduled restore point creation, exclude files from snapshot updates, etc. etc.), but feel I need to understand a lot more than I have been able to find documented... I'm going to blog the investigation so far shortly - the more facts and the less speculation and inference the better! Anything you can say or point to would be much appreciated.Anonymous
March 16, 2008
>> My problem is that on a Vista HP laptop (i.e. without the nominal benefits of Previous Versions in other SKUs) System Restore points are so large there is no useful Restore Point history... I am trying to gain a little control (reduce the frequency of scheduled restore point creation, exclude files from snapshot updates, etc. etc.), but feel I need to understand a lot more than I have been able to find documented... At the basic level you can think of the snapshot technology as operating at block level below the fie system - VOLSNAP.SYS is not aware of where the files are (except for the rare points in time when it needs to allocate the Shadow Storage files - which should be alocated in copy-free free space). So any change in the volume (whether it is file system metadata or changes in specific files) is tracked by VOLSNAP). You can see the overal shadow storage consumption through the VSSADMIN LIST SHADOWSTORAGE command. To reduce the overall shadow storage consumption you can reduce the maximum cap through VSSADMIN RESIZE SHADOWSTORAGE To reduce the amount of data per snapshot, you can exclude individual files through the registry key you mentioned. Now, that would work ony for small quantities of files (as the excluded files are deleted from the snapshot after the snapshot creation). If you have lots of files that generate churn (and therefore consumption of shadow storage) you coud try to move them on a separate volume. You can stil keep a pointer to these files from the original volume location (using mount points or, better, through symbolic links) so the namespace is kept the same.Anonymous
March 16, 2008
I was under the impression that in Vista it is not possible to have shadow copies located on a separate volume (I've tried).Anonymous
March 17, 2008
Many thanks for responding! I appreciate the file agnosticism of volsnap operations and the role of the VSS Optimizer in removing from the shadowcopies difference blocks relating to files that volsnap knows nothing about excluding. I understand how optimisation might not keep up with growth... I could reduce the shadowstorage area size but that's not the issue - the issue is that 15% of my notebook's 85GB C partition gives me only 4-5 days of SR points Re the FilesNotToSnapshot key, I created keys for "$UserProfile$* /s", "C:Program Files* /s", and "C:ProgramData* /s" - and most specifically "c:.tc /s") to exclude TrueCrypt containers (NB - lower case "c" - does that matter?] (A TrueCrypt container is an encrypted file mounted as a volume). Insofar as these keys match the pattern for the Vista default "Media Center record path" ("C:UsersPublicRecorded TV /s") these keys ought to prevent nearly all copy-on-write growth... but the initial file is so big it generally makes little difference with the following exception... After a basic snapshot of 3,565,158,400 Bytes (for ~160K files and ~18k folders) the file grew by ~700MB during the day in which I made ~700MB of changes to the TrueCrypt container. There is very little churn apart from on the TC volume. Do you have any idea why the TrueCrypt exclusion might not be working? The tc file is actually in a subfolder of Documents so should be caught by the /s folder recursion from root on its own exclusion, but also by the userprofile exclusion. [I have other backup strategies for docs etc.] Have I mis-specified the exclusions? The other questions relate to this: I created a restore point and monitored it. A file of 600 MiB (MiB = 1024^2B) was created immediately, and then after a delay of about 5 minutes it grew in increments of precisely 200 MiB every 40s or so until the file size was 3,400MiB at which point it stabilised, only to shrink again again by a a few hundred MB when the next snapshot occurred. Are you in a position to say why snapshots that initially contain no difference blocks are big? What's in there? And what happens in the two obvious writing phases? I can't find any explanation; are the details confidential perhaps? Best regards JulianAnonymous
March 18, 2008
The comment has been removedAnonymous
March 18, 2008
The comment has been removedAnonymous
March 19, 2008
The comment has been removedAnonymous
March 19, 2008
The comment has been removedAnonymous
March 19, 2008
The comment has been removedAnonymous
March 20, 2008
The comment has been removedAnonymous
March 22, 2008
Still planning on doing the difference test when I have got some restore points to compare... but here's an URGENT REQUEST in the meantime :( Just received by SP1 by WU and the System is now unstable & flaky (Support Request 1062217781 lodged) however, since I am in danger of losing the pre-SP1 restore point very quickly, can I separately backup the SVI folder and restore THAT later to do a System Restore? Thanks...Anonymous
March 23, 2008
Still in interested in whether SVI can be backed up and restored, but no longer critical... had to roll back with System Restore anywayAnonymous
March 23, 2008
The shadow storage files (stored in SVI) cannot be backed-up/restored individually. The only way to preserve your snapshots is to perform a block-level backup of the whole disk/volume (for example using a transportable HW snapshot). The reason is that the shadow storage contents have to be 100% synchronized with the contents of the rest of the volume (otherwise the differences will be innacurate). Thanks, AdiAnonymous
March 26, 2008
The comment has been removedAnonymous
March 27, 2008
Is there anyway to expose the most recent shadow copy via a script? Because the Vista backup utility is so useless, I plan to make a script that combines vshadow and robocopy to copy the folders that I want, and have that run daily. Since System Restore is already making a shadow copy each day, I'd rather tap into that one rather than force another one. Any idea on whether such a thing would be possible?Anonymous
March 27, 2008
Re Windows Defender & Restore Points... I've discovered there is an option (Advanced Options) to enable or disable the creation of Restore Points... strike "Other Snapshot Initiators"Anonymous
March 30, 2008
>>> I see (of course) that pagefile.sys exists within the mounted shadowcopy... do page writes really get edited out by the optimiser? Given the negligible impact VSS has on performance that's an amazing achievement if pagefile writes are really treated like everything else. Page file writes are not tracked as changes.Anonymous
March 30, 2008
>>> Is there anyway to expose the most recent shadow copy via a script? Yes - you can write one using a WMI script that enumerates shadow copies (using Win32_ShadowCopy), get the last one, and then expose it using one of the methods in this blog post. Note that a shadow copy is NOT a replacement for backup - you can lose shadow copies at any time due to extended writes.Anonymous
March 30, 2008
Right, if it doesn't find any, I can just dump a new one. I just didn't want to have to create one every day shortly after System Restore creates its own since I plan to have it as a scheduled task that runs each morning.Anonymous
May 20, 2008
I'm a big fan of being productive by not losing work. I don't care how optimized your system and development