Exchange / VSS / and differential block size…

Today’s backup and restore operations require close coordination between backup applications, the line-of-business application being backed up (for example, Exchange 2010), and the storage management hardware and software. The Volume Shadow Copy Service (VSS) in Windows Server 2008, which was first introduced in Windows Server 2003, facilitates the conversation between these components to allow them to work together. When all of the components support VSS, you can use them to back up your application data, such as mailbox and public folder databases.

 

VSS coordinates the actions that are required to create a consistent shadow copy (also known as a snapshot or a point-in-time copy) of the data that is to be backed up. Shadow copies use differential copy-on-write technology to maintain consistency during the lifecycle of the snapshot.  For more information on how this works, see https://en.wikipedia.org/wiki/Shadow_Copy and https://en.wikipedia.org/wiki/Snapshot_(computer_storage).

 

There are three primary VSS components: providers, requestors, and writers. The provider is the system-level component that performs the actual work of creating and representing shadow copies. The requestor is the backup application, such as Windows Server Backup, System Center Data Protection Manager, etc., that requests a backup from the provider. And the writer is application (or component within an application) that coordinates its I/O operations with VSS shadow copy and shadow copy related operations (such as backups and restores) so that their data contained on the shadow copied volume is in a consistent state.

 

Before choosing a VSS-based backup application for Exchange, I recommend that you check with the vendor to determine which provider their application (the requestor) uses, and specifically, what differential block sizes are used. This is important because the block size used impacts how efficiently storage will be utilized.

 

Let me explain why.

 

The built-in provider in Windows, which leverages VOLSNAP.sys, uses a differential block size of 16K.

 

 

If a snapshot has been created, VOLSNAP.sys begins to intercept writes to the volume.  If a single byte in 16K has changed, VOLSNAP.sys moves the original 16K to differential storage and allows the new write to proceed to the volume. 

 

 

If a write happens to span more than one 16K block then all blocks that are changed are moved to differential storage.

 

 

Let’s look at how this might impact an application like Exchange.  Exchange writes in a static page size. 

 

• Exchange 2003 – 4K
• Exchange 2007 – 8K
• Exchange 2010 – 32K

 

In the Exchange 2007 example, if a single write of 8K was located on a single 16K differential block size, only 16K is moved to differential storage.  If the 8K spanned two 16K differential blocks then 32K would be moved to differential storage. 

 

Most VSS-based backup applications leverage the built-in VSS provider.  However, it is not a requirement to do so, and some vendors instead implement their own providers.  In these cases, the vendor may also choose to implement a larger or smaller differential block size.

 

Let’s look at the example of a hardware-based provider that implements a VSS solution with a block size of 1024K.  In this example, if a single byte changes in a 1024K differential block, then 1024K of data is moved to differential storage.  For applications like Exchange that write data in static pages, this can have can negative consequences.

 

For example, Exchange 2007 writes an 8K page to a single 1024K differential block.  This results in 1024K being written to shadow storage.  Therefore, 1 MB of storage is used to store 8K of data change.

 

 

Worse, if the write of 8K spans two 1024K differential blocks this results in 2048K being written to shadow storage.  In this case, 2 MB of storage is used to store 8K of data change.

 

 

In some cases, the custom block size used by a vendor’s custom provider has a range of configurations.  And in some cases, it cannot be configured at all. Before implementing a solution, consult with the vendor to determine which provider they use. If they use a custom provider, verify that it works efficiently with your version of Exchange.

 

*I want to thank Scott Schnoll and Dennis Middleton for tech reviewing and editing this post.

Comments

• Anonymous
  January 01, 2003
  @Andrew: Thanks for the detailed question... To my knowledge networker utilizes the default software provider from Microsoft which would subject them to the 16 K limit. After a successful backup has completed to my knowledge the backup application will cleanup the shadows created.  This would then reclaim the space. You can see the shadows that you have created / existing by using VSSAdmin list shadows. You can also discard all shadows by running diskshadow (at the command prompt).  Then run delete shadows all followed by exit to return to the command prompt. TIMMCMIC

• Anonymous
  January 01, 2003
  Tim, This is very enlightening indeed. I think I see an inefficient block size being used by Networker to back up our Exchange 2010 databases. As I understand it, when writes are made to the database after a volume snapshot has been made, there are original blocks that are written to the differential storage which is located on a disk on the system in which the volume is located. In our case certain disks have been designated as the differential storage areas for other disks where the data volumes are located. Our issue now is that there is a great deal of space being eaten up by the differential data and of course we can't see it in the form of a file on the disk, but the space is definitely disappearing. Does this sound like something that could be attributed to what you are describing above? I will find out what the block size is that is being used by Networker. Also, looking at the vssadmin output  shows that the limit for the storage space is unbounded. I'm also wondering if that needs to be configured so that the space used is not too much. Will that create an issue if the storage space has a limit on it and the differential data goes over that amount? Any help understanding this completely is much appreciated. Thanks, Andrew

• Anonymous
  January 01, 2003
  When I look at the shadow details using diskshadow, I see the following for attributes.

• Attributes:  No_Auto_Release Persistent Differential Would you know what this means?

• Anonymous
  January 01, 2003
  The comment has been removed

• Anonymous
  November 07, 2013
  It is 2013  now so let’s name it a question from the past :-) , regarding the block size issue I think about it as an I/O vs. Disk Size, correct me if am wrong, does 1 MB block size mean more changes to the original 1 MB block will be handled per single snapshot provider copy?

• Anonymous
  November 08, 2013
  Hany: Nothing much has changed here.  A block is only moved to shadow storage one time - on the original change to the blog.  Subsequent changes to the block are allowed to proceed through to storage.  Remember - the goal is to ensure that we have a method to retain the original data set - so one move allows us to accomplish that. The issue outlined here still remains the same - if you have small data sets changing but larger block boundaries - you will end up bloating the amount of shadow storage space necessary to contain those smaller changes. TIMMCMIC

• Anonymous
  April 03, 2014
  What are orphaned vss shadow copies (viewable through hidden devices in device manager) and how they affect vss operations?