How Remote Storage Works
Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2
How Remote Storage Works
In this section
Remote Storage Terms and Definitions
Remote Storage Architecture
Remote Storage Protocols
Remote Storage Interfaces
Remote Storage Processes and Interactions
Network Ports Used by Remote Storage
Related Information
You can use Remote Storage to extend disk space on your server without adding more hard disks. Remote Storage automatically copies eligible files on your local volumes to a library of magnetic tapes or magneto-optical disks. Remote Storage then monitors the amount of space that is available on your local volumes.
When the amount of available space on a local volume falls below the level that you designate, Remote Storage automatically removes the content (data) from a sufficient number of eligible (or premigrated) files and migrates this content to an attached storage device, thus freeing up disk space on the volume. When data is removed from a file, the logical size of the file remains the same, and to the user, the file appears unchanged, but the physical size of the file is reduced to 1 KB. Content in other eligible files is not removed until more disk space is needed. When you need to open a file whose data has been removed, the data is automatically recalled from remote storage. To find more information about these processes, see “What Is Remote Storage?” in “What is Remote Storage?” See also “Remote Storage Terms and Definitions” and “Remote Storage Processes and Interactions” in this section.
Remote Storage runs on Windows Server 2003, Enterprise Edition; Windows Server 2003, Datacenter Edition; and the 64-bit versions of these operating systems. Remote Storage is also available on Windows?2000 Server., However, Remote Storage is not installed by default. To install Remote Storage, you can select Remote Storage during Setup, or you can later manually install Remote Storage through Add or Remove Programs in Control Panel. You can only manage Remote Server on NTFS volumes.
Levels of data storage
Remote Storage data storage is hierarchical, with two defined levels. The upper level, called local storage, includes the NTFS disk volumes of the computer running Remote Storage. The lower level, called remote storage, includes the automated media library or stand-alone tape or disk drive that is connected to the server.
Remote Storage supports all SCSI-class 4-mm tape, 8-mm tape, digital linear tape (DLT), and magneto-optical devices that are supported by Removable Storage. (To find more information about Removable Storage, see “What Is Removable Storage? To find more information about the relationship between Remote Storage and Removable Storage, see “Remote Storage Processes and Interactions” in this section.) Remote Storage does not support quarter-inch cartridge (QIC) tape libraries, rewritable compact disks (CD-RWs), or rewritable DVDs (DVD-RWs).
Retrieval of stored files
When you need to access a file on a volume managed by Remote Storage, you simply open the file as usual. If the data for the file is no longer on your local volume, Remote Storage recalls the data from a media library. Because this can take more time than usual, Remote Storage removes the data only from those files on your local volumes that you are least likely to need, based on criteria that you set.
Coordination with other tools
Remote Storage uses Removable Storage to access the applicable tapes contained in libraries. Remote Storage also works with Backup for data recovery, and with Task Scheduler to schedule file copy operations. For information about Removable Storage, see “Removable Storage Technical Reference”. For information about Backup, see “Backup Technical Reference.”
Remote Storage also provides certain data-recovery features, including the ability to generate multiple copies of data in remote storage.
Business Benefits
Remote Storage provides several significant business benefits, including the following:
A low-cost solution for archiving files that are seldom accessed, but should still be available.
A low-cost solution that enables virtual expansion of local storage space.
Transparent automatic access to data in remote storage.
Automation of the labor-intensive overhead associated with daily manual data-management operations.
Centralized sharing of remote storage for multiple volumes.
Note
- Using Remote Storage to copy files to remote storage is not the same as backing up your files. You should follow a regular schedule of data backups, including backing up the contents of the local volumes that Remote Storage manages. You should also back up the Remote Storage database and other program files located in the System32\RemoteStorage folder.
Notes
Do not create File Replication service (FRS) replica sets on a volume that is managed by Remote Storage. In addition, do not add a volume to Remote Storage that contains directories that are part of an FRS replica set. Otherwise, you might severely impact system performance and possibly cause data loss within your media library.
FRS might need to periodically read every file in the replica set to send the file contents to another computer. This causes FRS to recall all files that Remote Storage has sent to secondary storage, which might take a long time (hours or days). If you use tape for your secondary storage, remember FRS recalls files in directory order rather than media order, so the excessive number of tape seeks performed by FRS will likely ruin the tapes and cause data loss.
The following sections provide an in-depth view of how Remote Storage works in an optimal environment. An optimal environment for Remote Storage is defined as follows:
Windows Server 2003 is correctly installed and Remote Storage is enabled.
All other dependencies are in place, properly designed and deployed, and functioning normally.
Server hardware is sized appropriately and there are no disk, CPU, memory, or network bottlenecks that affect the performance of the technology.
Remote Storage Terms and Definitions
The following terms are associated with Remote Storage*.*
Managed volume
A local volume formatted with NTFS file system 5.0 whose disk space is managed by Remote Storage. Remote Storage frees up disk space by automatically moving infrequently accessed files to a remote storage device.
Desired Free Space
The amount of free space that you specify a volume should have at all times during normal use.
Eligible file
A file that has been premigrated, and is therefore ready to be truncated.
File selection criteria
User-defined criteria that determine which files Remote Storage will premigrate. One such criterion might be that all files that are .doc files should be premigrated.
Unnamed data attribute
The default data stream of an NTFS file, sometimes referred to as $DATA. The unnamed data attribute is copied to remote storage during premigration.
Premigrated file
A file whose unnamed data attribute has been copied to a remote storage device in preparation for truncation. Until truncation and migration take place, this file remains unchanged on the managed volume.
Truncated file
A file on a managed volume from which Remote Storage has removed data so that the physical file size is reduced to 1 KB. This file appears unchanged to the user.
Migrated file
A file that Remote Storage has copied from local storage to remote storage. The resulting file in remote storage is not visible to users. The file remaining in local storage is truncated.
Automatic File Truncation
The process in which data is removed from a sufficient number of premigrated files when available disk space on a managed volume falls below the required level. This process occurs without user intervention.
Scheduled File Truncation
The process in which a user forces data to be removed from premigrated files even if available disk space has not fallen below the required level.
Placeholder
An identifier for a file that has been migrated from local storage to remote storage. Placeholders are not visible to users.
Remote storage
The term “remote storage” (lower case) refers to infrequently accessed data and the media on which it is stored until the data is needed again.
Removable Storage
A service that you can use to manage removable media (such as tapes and discs) and storage devices (libraries). Removable Storage enables applications to access and share the same media resources. For a complete listing of Removable Storage terms and definitions, see MSDN and type Removable Storage Glossary in the “Search for” text box.
Remote Storage Architecture
From an architectural standpoint, Remote Storage is divided into three areas:
Remote Storage components: The Remote Storage Microsoft Management Console (MMC) snap-in and the Remote Storage service.
Windows components and services: Win32 tape and disk APIs, tape and disk drivers, changer drivers, Event Viewer, and the registry.
Libraries: A robotic or automated library, stand-alone tape drive, and stand-alone disk driver, for example.
The following figure shows the components within these three areas and how the areas and components interact.
Remote Storage Architectural Diagram
.gif "Remote Storage Architectural Diagram")
The following table lists the components that are included in Remote Storage.
Remote Storage Components
| Component | Description |
|---|---|
Remote Storage MMC snap-in |
Administrative interface to Remote Storage. The Remote Storage snap-in enables you to:
|
Remote Storage service |
Two system services that implement Remote Storage functionality. The primary service starts automatically. The second service is used for recall notifications. |
The following table lists the items that are included in Windows components and services.
Windows Components and Services
| Service | Description |
|---|---|
Win32 tape and disk management APIs |
Comprises four different APIs: CreateFile, DeleteFile, GetDiskFreeSpace, and GetDiskFreeSpaceEx. To find more information about these APIs, see “How Removable Storage Works” in “How Removable Storage Works.” |
Tape and disk driver |
Enables a specific device, such as a stand-alone tape or disk drive, to communicate with the operating system. Although a device might be installed on your system, Windows cannot use the device until you have installed and configured the appropriate driver. If a device is listed in the Windows Catalog, a driver is usually included with Windows. Device drivers load automatically (for all enabled devices) when a computer is started, and thereafter run in a manner transparent to users. (The Windows Catalog website has hardware and software compatibility information for the Windows 2000 Server family, Windows XP, and the Windows Server 2003 family.) |
Changer driver |
Enables a specific device, such as an automated or robotic library, to communicate with the operating system. Although a device might be installed on your system, Windows cannot use the device until you have installed and configured the appropriate driver. If a device is listed in the Windows Catalog, a driver is usually included with Windows. Device drivers load automatically (for all enabled devices) when a computer is started, and thereafter run in a manner transparent to users. |
Event Viewer |
A component that you can use to view and manage event logs, gather information about hardware and software problems, and monitor security events. Event Viewer maintains logs about program, security, and system events. Recalls of files from Remote Storage are logged in the Windows event log. |
The registry |
A database repository for information about the configuration of a computer. The registry contains information that Windows continually references during operation, such as:
The registry is organized hierarchically as a tree and is made up of keys and their subkeys, hives, and entries. |
The following table lists the components that are included in libraries.
Libraries
| Library | Description |
|---|---|
Robotic or automated library |
Automated units that hold multiple tapes or disks. Some have multiple drives. These libraries are sometimes called changers or jukeboxes, and commonly use robotic subsystems to move media stored in the library’s storage slots. Automated libraries can also consist of other hardware components that are managed by Removable Storage, such as doors, inject/eject ports, cleaner cartridges, and bar-code readers. |
Stand-alone tape drive |
A single-drive, nonautomated unit that holds a single tape. With this device, you manually insert a tape into the unit. |
Stand-alone magneto-optical disk drive |
A single-drive, nonautomated unit that holds a single disk. With this device, you manually insert a disk into the unit. |
Remote Storage Protocols
Remote Storage does not utilize any networking protocols.
Remote Storage Interfaces
There are no Remote Storage interfaces available.
Remote Storage Processes and Interactions
The following is a general description of how Remote Storage functions at a basic level. (To find more information about the terms used in this section, see “Remote Storage Terms and Definitions” earlier in this section.)
Remote Storage is a hierarchical storage management application that migrates data from upper-level storage to lower-level storage. Upper-level storage is commonly known as local storage: frequently accessed data that is stored locally on high-performance disks. Lower-level storage is also known as remote storage: infrequently accessed data that is stored on less expensive media until it is needed again. Hierarchical storage management reduces the costs of storing large quantities of data, while ensuring that the data is still accessible. The following table outlines when to use the two levels of Remote Storage.
Levels of Remote Storage
| Considerations | Upper-level Storage | Lower-level Storage |
|---|---|---|
Storage type |
Local or primary |
Remote storage |
How often is data needed? |
Frequently |
Less frequently |
Media type |
High-performance disks |
CD-ROMs DVD-ROMs Magnetic tape, which includes:
Magneto-optical disk drive |
Storage cost |
More expensive |
Less expensive |
Regardless of where you store the data, the file system namespace continues to provide users with access to the file. For example, when a user double-clicks a file in My Documents, Remote Storage retrieves the file from its storage location and restores it to the file system.
For users to access files that are stored on offline media, the media must be placed back online before the files are accessible. Therefore, these files require more time to access than files that are stored on online media.
The following is a more in-depth examination of how Remote Storage works at a basic level.
To prepare for the need to free storage space on a local server, Remote Storage regularly premigrates (copies) the unnamed data attribute of all eligible NTFS files on local storage to remote storage. After this is done, the files on local storage are considered to be premigrated. The unnamed data attribute of an NTFS file contains information that is tracked by NTFS. This information is needed to manage the data. To find more information about this process, see “What Is Remote Storage?” in “What Is Remote Storage?”
When the amount of available space on a local volume falls below the Desired Free Space, Remote Storage automatically removes the content (data) from a sufficient number of eligible files and migrates this content to an attached storage device, thus freeing up disk space on the volume. This process is referred to as truncation. When data is removed from a file, the disk space used by the file is reduced to 1 KB, although the file itself appears unchanged. At the same time, a “placeholder” is generated for each such file and this placeholder serves as an identifier for its newly-created counterpart on the attached storage device. Remote Storage marks the placeholder with the flag, FILE_ATTRIBUTE_OFFLINE. Content in other eligible files is not removed until more disk space is needed. When you need to open a file whose data has been removed, the data is automatically recalled from remote storage.
Even though Remote Storage changes the physical size of the file on local storage, the logical size and the date and time (created, last modified, last accessed) of the file remain unchanged. Users or applications (for example, disk quotas) that use file size as a determinant of a particular action are not affected by Remote Storage data migration.
For these processes and interactions to work as described, Removable Storage should reside in an optimal environment. An optimal environment for Removable Storage is defined as follows:
Windows Server 2003 is correctly installed and the Removable Storage service is running.
All other dependencies are in place, properly designed and deployed, and functioning normally.
Server hardware is sized appropriately and there are no disk, CPU, memory, or network bottlenecks that affect the performance of the technology.
Remote Storage Technologies
The following is a more detailed description of specific technologies, features or attributes that either form a part of, or are closely identified with, Remote Storage.
Removable Storage
Remote Storage uses Removable Storage for copying data to online libraries that contain removable media. For example, if a volume that is being managed by Remote Storage meets the requirements for migrating data to secondary storage, Removable Storage moves media into a slot so that Remote Storage can migrate data to that media. To find more information about this topic, see “How Removable Storage Works" in “How Removable Storage Works.”
Disk Management Attribute
In Remote Storage, you can define a single, system-wide, disk management attribute that can be applied to each managed volume. This attribute is referred to as the Desired Free Space setting and establishes a threshold that triggers Automatic File Truncation. The purpose of Automatic File Truncation is to free up disk space on a managed volume when the amount of space falls below a certain percentage. The default setting is five percent, but this can be adjusted to any percentage that you specify. In addition, you can set the level of Desired Free Space for several different managed volumes all at the same time.
Notes
It is important to realize that although the Desired Free Space setting is used to determine when files that have already been premigrated should be truncated, it is not used to determine how many files should be premigrated before the need for truncation occurs.
In addition, by default, Remote Storage does not premigrate the Windows system files (or any files marked with the system attribute).
Premigration
Files are selected for premigration based on the following criteria:
Path name
File names, including wildcard characters in names
Inclusion or exclusion of premigration specifications
The following criteria can be applied to an entire volume, but not to an individual file:
Last access date or inactivity period
Owner
File size
System and hidden files
You use these criteria to specify which files will be premigrated and, consequently, become eligible files. Eligible files are files that are truncated as the need for additional disk space arises. Remote Storage premigrates all files that meet the criteria and tracks these files to allow rapid file truncation. Even though Remote Storage changes the physical size of the file in local storage, the logical size and the dates and times (created, last modified, last accessed) of the file remain unchanged.
Remote Storage applies the inclusion and exclusion rules in such a manner that the more specific a rule, the higher its priority. For example, the rules of “Exclude \*.*” and “Include \PROG\*.EXE” causes the file “\Prog\Test.exe” to be included and “\Test.exe” to be excluded.
Automatic File Truncation
Remote Storage initiates Automatic File Truncation whenever the level of free space in a managed volume becomes less than the Desired Free Space. Eligible premigrated files are truncated to free space on local storage. The premigrated files are quickly identified and truncated, based on the last access date. The file truncation process continues until the free space on the volume is greater than the Desired Free Space.
If the premigrated file has not been modified, then the file is truncated and a placeholder for the file is created. If the premigrated file has been modified, the file is no longer considered premigrated and is returned to “normal file” status. In addition, the capacity flow meter is decremented and the data for the premigrated file in remote storage is marked as deleted to facilitate space reclamation. Reclamation is the process of deleting from the Remote Storage media pool any files that are no longer necessary, thereby freeing up additional storage space.
Note
- Storage maintains a flow meter capacity metric that monitors the amount of data that is contained in remote storage, including overflowed and shelved data. Copying data from local storage to remote storage increases the value of the capacity metric. Removing local storage references to data in remote storage decreases the value of the capacity metric.
Scheduled File Truncation
Scheduled File Truncation is the process in which a user forces data to be removed from premigrated files even if available disk space has not fallen below the required level. Premigrated files are checked to make sure that they have not been modified since they were premigrated. If the premigrated file has been modified, then the file is not considered premigrated and the file is returned to normal file status. Additionally, the data for the premigrated file in remote storage is marked as deleted to facilitate space reclamation.
You can use Scheduled File Truncation to manage disk space proactively. The action can be scheduled to reduce volume usage in advance of a known volume-intensive event, such as the installation of a very large application.
Validation of Managed Files
You can use Validate Managed Files to validate premigrated files and placeholders, thus making sure that all the files on your managed volumes point to valid and correct data in remote storage. Validation includes detecting placeholders moved between volumes, disconnected placeholders, and modified premigrated files. Corrective actions are taken immediately to synchronize the placeholders and premigrated files with remote storage. Validation also recalculates volume statistics. Validation is automatically performed two hours after a backup program is used to restore a remote storage file, but can also be scheduled manually.
You should perform validation a regular basis to validate all files on a given volume and to correct any inconsistencies. You should also perform validation after restoring files on a local volume or after disk errors have occurred on a volume.
File Recall
The file is recalled on the first read, write, or memory map request. If the data is on tape or disk in remote storage, but the tape or disk is not in the library (storage device), the recall does not succeed and returns the message STATUS_FILE_IS_OFFLINE. In order to successfully recall the file, you need to manually insert the tape or disk into the library and then try the recall again.
Notes
Remote Storage does not affect a request that is made against a premigrated file. The data for the file exists locally on the NTFS volume and can be accessed normally. However, the client making the request on the server might time out. Client computers have their own configured time-out that is independent of Remote Storage, and if accessing the data takes longer than the time-out, the input/output request is unsuccessful for the client.
Changing attributes, access control lists (ACLs), or named streams in a file does not recall the file.
Runaway Recall Detection and Prevention
Runaway recalls can be prevented by specifying that no more than a certain number of files can be recalled by a user from a volume in an hour. An administrator defines this setting, both in terms of enabling or disabling the feature, and in specifying the number of recall requests that are handled in a particular period of time.
Renaming, Moving, or Copying Placeholders and Premigrated Files
Migrated files (and their associated placeholders) and premigrated files can only be renamed on the same volume. Files cannot be renamed across volume boundaries. It is important to note that renaming a file does not cause the data in remote storage to be recalled.
However, it is possible to copy or move placeholders across volume boundaries. When this occurs, the data in remote storage is recalled. At the end of a move operation between volumes, the original placeholder is deleted and the file information that has been recalled and is now present on the other volume once again assumes a premigrated status. At the end of a copy operation between volumes, the original placeholder is also deleted, but there now exist two files, the first on the original volume and the second on the other. The same process applies when copying a placeholder to a different location on the same volume: the data in remote storage is recalled, the original placeholder is deleted, and there now exist two identical files, both on the same volume.
Note
- In order to prevent file information from being recalled when moving or copying a file to a different volume on the same computer, you should use Backup to back up and restore the file to its new location, rather than moving or copying the file directly. Despite being moved or copied, the file will continue to point to the correct location in remote storage.
Discontinuing Volume Management
If necessary or desirable, you can specify that Remote Management no longer manage a volume. In doing this, you are given two choices:
Remote Storage discontinues managing the local volume, but leaves data in remote storage. Data that is already in remote storage can be recalled as before. No automatic freeing of disk space takes place.
Remote Storage recalls data from remote storage, and then discontinues managing the local volume. You should only select this option if your local volume has adequate disk space for the recalled data.
If you have adequate disk space to recall data from remote storage, but the option of recalling all data is disabled, click the Refresh button in order to update volume statistics. After you have updated the volume statistics, the option should be enabled.
Restoring Placeholders from Backup and Disconnected Placeholders
The data in remote storage is accessible only through a placeholder. If the placeholder is deleted, that data is inaccessible. An upper-level storage backup application, such as Backup, protects placeholders. If the placeholder is deleted, lost, or destroyed, the only recourse is to retrieve a copy from the backup media. If a placeholder is restored from backup, the placeholder is reconnected to that data. This is the same mechanism that you must use if any other file on the volume is deleted, lost, or destroyed.
Disconnected placeholders are placeholders whose file contents have been removed from remote storage. These placeholders might have been restored from backup after the space in remote storage was reclaimed, or the data within remote storage might be physically unavailable because of media failures. Disconnected placeholders do not point to valid locations in remote storage. By validating placeholders, you remove disconnected placeholders from the system and synchronize the remaining placeholders with remote storage.
Interaction with Other Reparse Point Types
Volume mount points and symbolic links cannot be changed into placeholders. The system explicitly ignores these types of files. Any file system scan that encounters a mount point or symbolic link does not follow the link.
Placeholders Binding to a Specific Remote Storage Engine
Placeholders identify the specific remote storage system that contains the managed data, binding a placeholder to a specific Remote Storage engine. Placeholders cannot be moved between multiple remote storage systems.
Windows Server 2003 Family Security Integration
Remote Storage supports NTFS security features. Remote Storage recalls files only if the user has valid access to a placeholder. Because a placeholder is identified as a special remote storage reparse point, the security of the reparse point mechanism determines the security of the placeholder.
Administrators and users undergo the normal security checks for Windows users. The Remote Storage Administrative user interface uses Windows permissions to grant or deny access to the interface. Only user accounts in the Administrators group are allowed access to the UI functions.
Grouping of Files
Files are grouped together by their volume and by the time at which they were originally premigrated. The files remain grouped during all remote storage operations. No other grouping of files is supported.
Replication
By using the file selection criteria, the administrator can exclude from the Manage Files action any files that are being replicated.
Remote Storage Engine
Remote storage is the part of the system that holds the copies of the data. The Remote Storage engine consists of libraries, drives, and the media used to store the data.
The Remote Storage engine service runs only on computers that are running Windows Server 2003, Datacenter Edition; Windows Server 2003, Enterprise Edition; or Windows 2000 Server. Only one Remote Storage engine can exist on a server.
Flow meter capacity metric
Remote Storage maintains a flow meter capacity metric that monitors the amount of data that is contained in remote storage, including data that has overflowed onto other media or is located on media that are no longer in the device and have been shelved. Copying data from local storage to remote storage increases the value of the capacity metric. Removing local storage references to data in remote storage decreases the value of the capacity metric.
Remote Storage increases the value of the flow meter capacity metric whenever one of the following occurs:
A new file is managed.
A previously deleted placeholder is restored to a volume and reconnected to remote storage.
Remote Storage decreases the value of the flow meter capacity metric whenever one of the following occurs:
A premigrated file is modified.
A placeholder or premigrated file is deleted.
A file is unmanaged.
There might be a delay in changing the value of the flow meter capacity metric because detection of these conditions can happen at a later time.
Media pool for remote storage
All remote storage in Remote Storage exists in a single storage or media pool. A media pool is defined as a unit of storage that is administered by Removable Storage and composed of homogeneous storage media. A media pool is a self-contained storage area with homogeneous characteristics — for example, random access, sequential access, read/write, and write-once. To find more information about media pools, see “How Removable Storage Works” in “How Removable Storage Works.”
Remote Storage supports one storage library within the single storage pool. The library must have at least one drive. To support media copies, it must have two or more drives. When multiple drives exist within a library, all drives must be identical in type.
Note
- Relocation is the movement of data from one remote storage pool to another. Because Remote Storage only supports a single remote storage pool, relocation is not supported in Remote Storage.
Handling remote storage that is full
Remote Storage provides the following options for handling situations in which remote storage is physically full:
Stop managing new files. Remote Storage becomes a recall-only system. This mode is invoked only after any possible space reclamation is performed.
Shelf storage. Remote Storage cycles removable media to shelf (offline storage). Recalls of files stored on shelved media require physical intervention.
Remote storage is considered full either when there is not enough space on the media or when the media are not online. In either case, you must physically intervene to allow Remote Storage to continue migrating files. A notice is issued when physical remote storage use reaches the warning level that is defined by the administrator.
Shelf media for Remote Storage
Shelf media for Remote Storage are media that are not currently accessible by the system. The medium with the data on it can be removed from the device and the media management subsystem tracks its location. Shelf media allow a system to have more data in remote storage than the device can physically contain.
For removable media, you can remove media and optionally replace them with blank media. Remote Storage continues to premigrate data by using the new media. The removed media are considered shelf media. If a user needs access to data that is on shelf media, you must place the shelf media in the library, and the new media become shelf media. Media do not need to be full to be shelf media.
Upgrading Remote Storage libraries
Removable Storage provides Remote Storage with the ability to replace a library with another library that contains identical media and data read/write mechanisms. You must physically move the media from the old library to the new library, that is, physically replace the library, and then use the Remote Storage snap-in to register configuration changes within Remote Storage.
Remote Storage Device Support
Remote Storage supports all of the same 4-mm, 8-mm, and digital linear tape (DLT) and magneto-optical disc devices that Removable Storage supports. Because Remote Storage supports only a single media type, a device cannot be a mixed media device.
Removable Storage uses Windows kernel-mode drivers to support the management of tape and optical disc devices. These device drivers provide the backbone of device support for both Removable Storage and Remote Storage. The Windows Logo program identifies device drivers that have been approved for use with the Windows Server 2003 family.
Remote Storage cooperatively shares storage libraries with external applications by using a common media management facility, such as Removable Storage.
Note
- Remote Storage does not support using small-capacity removable disk devices such as the Iomega Zip drive.
Remote Storage Integration in Windows Server 2003
Remote Storage is integrated closely with the NTFS file system in Windows Server 2003, Enterprise Edition and Windows Server 2003, Datacenter Edition. The following table provides details of this integration and describes the interfaces between Remote Storage and these operating systems.
Remote Storage and Windows Server 2003 Integration
| Interface | Description |
|---|---|
Placeholder format and reparse points |
A placeholder has the system-defined $REPARSE_POINT attribute set with information that can identify and retrieve the unnamed data attribute from remote storage. The unnamed data attribute is in remote storage and its length is zero bytes. The reparse point type is “Remote Storage”. The recall mechanism is based on the reparse point being identified by NTFS and is an efficient mechanism that uses this information when recalling a file. |
File size of a placeholder |
The logical file size of the unnamed data attribute of a placeholder is the same size as it was before it was truncated. The physical size of the unnamed data attribute of a placeholder is zero bytes. |
Disk quotas and remote storage |
Disk quotas do not change. Disk quotas are based on the logical file size that is not changed by Remote Storage. Therefore, the amount of space in use on a volume is not reduced or changed by having data in remote storage. |
Remote Storage stage change |
Remote Storage supports a Distributed Component Object Model (DCOM) interface–based mechanism that can change the system status. |
Event log usage |
Major Remote Storage system events are logged in the Windows event log. You can use the Windows Event Viewer to see the logged events. The following events are optionally logged in the Windows event log:
|
Windows Server 2003 Job Scheduler |
Remote Storage uses the Windows Server 2003 Job Scheduler to schedule Remote Storage jobs. Job status can be monitored from within the Remote Storage snap-in. Depending on the Job Scheduler, a job window can be specified that limits the amount of time spent in a single scan. This is useful because the system might have a large amount of data to manage and the scan might initially take too much time. A bookmark is kept where the scan stopped so the scan can continue from that point the next time it is run. |
Windows Server 2003 registry usage |
Remote Storage uses the registry to keep persistent information about startup, including the programs needed to initialize Setup. |
Windows Server 2003 client time-out on recall |
The registry entry HKEY_LOCAL_MACHINE\SYSTEM \CurrentControlSet\Services\LanmanWorkstation\parameters \OfflineFileTimeOutIntervalInSeconds controls the time-out period that the client uses when sending requests to a server. The default is value is 900 (or 15 minutes). You can increase this value on all Windows clients that open files on the volumes managed by Remote Storage. |
Multiple Remote Storage Interactions
A Remote Storage installation is limited to a single system running Windows Server 2003, Enterprise Edition or Windows Sever 2003, Datacenter Edition. There is no limit, however, to the number of stand-alone Remote Storage instances that can run on a homogeneous Windows network.
Moving data from one Remote Storage installation to another
To move managed data from one Remote Storage installation to another, you recall managed data and move the data to another volume managed by Remote Storage.
Choosing Products That Work with Remote Storage
Because it manages data transparently for applications and users, Remote Storage does not require that the other applications running on NTFS be completely compatible with it, nor does it require that the applications contain specific code to recognize that Remote Storage is running. However, products that regularly open many files might cause a great deal of data recall, thus reducing the benefits provided by Remote Storage. Thus, for best results, use applications that are compatible with Remote Storage.
Note
- Although the features included with Windows Server 2003, Enterprise Edition and Windows Server 2003, Datacenter Edition are compatible with Remote Storage, many other applications are not and can result in time-outs or runaway recalls. In all cases, check with your independent software vendor (ISV) in order to determine if they provide a version that is compatible with Remote Storage.
Backup Programs and Remote Storage
A backup product that is compatible with Remote Storage must perform the following actions:
Guarantee that a full backup of the file is kept. The backup product must recognize the difference between backing up the placeholder and backing up the full data of the file, and must keep the full backup copy.
Back up a placeholder (reparse point) correctly.
Use the FILE_FLAG_OPEN_BACKUP_SEMANTICS parameter when opening a file to back it up.
Properly use FILE_FLAG_OPEN_NO_RECALL.
Backup applications that are compatible with Remote Storage protect placeholder data and do not cause recalls.
Remote Storage relies on the primary backup to protect placeholder data. You must be aware of when placeholder backups occur. Specifically, if backup rotations are used, you must know when the primary backup databases contain a complete copy of the data and when they contain placeholder data.
Backup applications that open placeholders to read and copy without using the FILE_FLAG_OPEN_BACKUP_SEMANTICS parameter cause the data for the file to be recalled. If these products are performing a full system backup, all placeholders are replaced with recalled data.
Antivirus Programs and Remote Storage
Antivirus programs typically open files to check for infection by viruses. These products sometimes cause migrated data to be recalled. Antivirus programs that are compatible with Remote Storage can operate in one of the following ways:
Detect placeholders and skip the check on such files. This assumes that the scanner checked the data before premigrating it. This might be somewhat hazardous because Remote Storage manages the unnamed data stream and does not manage named data streams. Because data in remote storage remains unchanged, an additional check for viruses is unnecessary.
Open the file with the FILE_OPEN_NO_RECALL parameter. This allows the scanner to check the unnamed data stream by using local storage without recalling the data. This assumes a linear search of the file data, which might not always be the case.
Do not update the last access date and time.
Document Management Programs
Many document management products open the data that they are managing, scan for key data, and then store references in additional files or alternate data streams, leaving the original data unchanged. A document management product that is compatible with Remote Storage can open placeholders with the FILE_OPEN_NO_RECALL parameter, which allow the migrated unnamed data to be read but not recalled.
In addition, after the index files are generated, some document management products provide a file viewer much like Windows Explorer. If the product is compatible with Remote Storage, when it opens each file, it experiences synchronous recalls of each file and any associated delay. The product also indicates which files are migrated and allows the user to issue asynchronous recalls of the data, while exhibiting improved performance.
Policies for managing files of the document management system exclude index files and other special files so that they are quickly accessible to the document management system.
Note
- The Indexing service that is included with the Windows Server 2003 family is compatible with Remote Storage.
Remote Storage Protection and Recovery
Media in remote storage are copied to protect data. Copies are updated as the original media are updated to ensure that a current copy of data is available.
Media copies
Media copies are replicas that can be substituted for the original media. Data is migrated only to the original media, but can be recalled from either an original or a copy.
Copies are made before the original media are completely filled and are updated as the original is updated. When a media copy is complete, it can be removed from the library that is located with other media in the copy set, and a new media copy is started.
Remote Storage creates media copies only if a library has two or more available drives. Other media, for which copies cannot be made, must be protected by different means, such as strict physical or environmental safeguards. To replace an original with a copy, you must manually perform the action. You should uniquely identify media copies in order to avoid confusion among different versions.
When a media copy is used to replace damaged or lost remote storage media, Remote Storage automatically copies the media again to ensure a complete set of copies.
Partial media copies are made by the system to protect the data. These partial copies can be removed from the library and are updated when they are reinserted in the library. If the source of the media copy is on the shelf, the source media must be reinserted in the library for the completion of the media copy.
Copy sets and set rotation
Remote Storage supports up to three copy sets of remote storage media. Media for the additional sets can be moved to shelf storage when they are synchronized. You can view the status of media copies.
Media for media copies
If a tape library has two or more available drives, Remote Storage uses scratch media for making a media copy. You can remove the updated media copies after several new media copies have been made. This is especially useful on weekends, during vacations, or following initial startup. After a copy is made, it must be returned to the library to be updated.
Metadata Protection
Remote Storage metadata exists in files that can be protected by the primary backup system (Backup) for Windows Server 2003, Enterprise Edition and Windows Server 2003, Datacenter Edition. Windows Server 2003 file security ensures that only the account used by Remote Storage has access to the metadata. It is therefore necessary for the administrator to assign appropriate user rights to this account so that only certain individuals (or groups) have access to this metadata. Otherwise, the metadata might be at risk, and if deleted, would no longer be retrievable.
Network Ports Used by Remote Storage
Remote Storage does not utilize any network ports.
Related Information
The following resources contain additional information that is relevant to this section.
Resource Kit Tools Help in the Tools and Settings Collection.
Command-Line References in the Tools and Settings Collection.
Group Policy Settings Reference in the Tools and Settings Collection.
To find more information about Removable Storage interfaces, see MSDN and type “Disk Management Functions” in the “Search For” text box.