Thin Provisioning

Overview

Thin provisioning is a storage management technique that optimizes the allocation of physical storage resources. It allows administrators to allocate larger amounts of virtual storage to applications than is physically available, with the expectation that not all allocated storage will be used at the same time.

Thin provisioning is an end-to-end storage provisioning solution that offers just-in-time allocations. It requires planning for storage deployment and execution on the host and client application.

Windows Server thin provisioning features serve as interfaces between the thin provisioning-capable storage and the host server. Thin provisioning features include:

• Thin provisioning logical unit (LUN) identification
• Threshold notification
• Handles for resource exhaustion
• Space reclamation for delivering highly available and scalable storage provisioning service to end users

Thin Provisioning LUN Identification

Windows Server adopted the T10 SCSI Block Command 3 (SBC3) standard specification for identifying thin provisioning logical units (LUNs) starting with Windows Server 2012.

During the initial target device enumeration, Windows Server gathers all property parameters from the target device, and identifies the provisioning type and the UNMAP and TRIM capability. The storage device reports its provisioning type and UNMAP and TRIM capability according to the SBC3 specification.

If the storage device doesn't accurately report its current capability, device compatibility issues might occur. For example, if the storage device reports that it supports the UNMAP command but doesn't support the UNMAP command, a disk format hang issue might occur. When provisioning type information is accurate, the storage stack can provide better I/O handling according to the storage provisioning type.

Run-time Provisioning Type or LUN Capacity Changes

A storage administrator can change the provisioning type or the capacity of the LUN. When the provisioning type or LUN capacity is changed, the storage array raises a UNIT ATTENTION sense condition that returns the correct information when the sense data is requested. Windows Server logs a system event to alert the system administrator of the provisioning type or LUN capacity change.

Threshold and Resource Exhaustion Handles

A thin provisioning LUN is usually created with less physical disk space than the size of the LUN. Threshold notification is a required function to alert the host and client application of the storage space consumption state.

Most thin provisioning storage arrays don't report an event when the threshold is reached. These thin provisioning storage solutions resolve the threshold notification through their proprietary storage management utility. Therefore, for the host and client application, the only event that such storage arrays report is permanent resource exhaustion. The thin provisioning storage device can use any of the following handles to alert system administrators or client applications when storage space consumption is nearing capacity:

• The threshold notification handle.
• The temporary resource exhaustion handle.
• The permanent resource exhaustion handle

Thin Provisioning Threshold Notification

The storage management utility sets the thin provisioning threshold. Windows Server doesn't override the threshold set by the storage management utility. For the thin provisioning LUN, the storage administrator must specify the threshold according to the average storage consumption rate. When a write command crosses the threshold set by the storage target device, the target device terminates the command by using sense data and sends a “THIN PROVISIONING SOFT THRESHOLD REACHED” message. When Windows Server receives the matched sense data, the following occurs:

• A system event is logged to alert the host administrator that the resource usage or availability threshold was reached on the LUN device.
• Information about the used and available mapped resources is reported in the system event log from the log page of the target device. For this reporting to occur, the storage array must support the log page specification for logical block provisioning in order for Windows Server to generate the system event.
• The terminated command is retried.

Write commands sent after this error is logged could potentially be lost if FILE_FLAG_WRITE_THROUGH isn't set because they might trigger a permanent resource exhaustion condition.

Temporary Resource Exhaustion

When the storage array enables the auto grow function on a LUN, an administrator can use the temporary resource exhaustion notification to ensure that the storage device can allocate more space to the LUN within four seconds. When a write command causes a temporary resource exhaustion condition, the storage device terminates the command that requests the operation by using sense data and returns a “SPACE ALLOCATION IN PROGRESS” message. Temporary resource exhaustion is handled as follows:

• Retry the original request four times with the retry interval set to 1 second.
• If all retries fail, the request is failed back to the application.
• If the storage device doesn't handle the temporary resource exhaustion, Windows Server expects the storage device to fail the next write request by returning a permanent resource exhaustion status.

Permanent Resource Exhaustion

The permanent resource exhaustion condition indicates that the thin provisioning LUN reached the maximum storage space limit. When permanent resource exhaustion occurs during a write command, the storage device terminates the operation by using sense data and sends a “SPACE ALLOCATION FAILED WRITE PROTECT” message. Permanent exhaustion is handled as follows:

• The write command is failed back to the application if the command has FILE_FLAG_WRITE_THROUGH set.
• The application might receive a success response without the request being completed nor flushed to physical media if the write command doesn't have FILE_FLAG_WRITE_THROUGH set.
• A system event is logged that includes a "permanent resource exhaustion" error message.
• The error code is passed back to the partition manager and the LUN is taken offline.

Storage Space Reclamation Using the UNMAP Command

The following scenarios can trigger space reclamation:

• A file deletion.
• A file system level trim.
• A storage optimization operation.

File system level trim is enabled for a storage device designed to perform “read return zero” after a trim or an unmap operation.

Space Reclamation Operation in the Storage Stack

Windows Server converts file delete or trim notifications into a corresponding UNMAP request:

• When a large file is deleted from the file system.
• When a file system level trim is triggered.

The storage port driver stack translates the UNMAP request into a SCSI UNMAP command or an ATA TRIM command according to the protocol type of the storage device. During the storage device enumeration, the Windows storage stack gathers information about whether the storage device supports UNMAP or TRIM commands. Only the UNMAP request is sent to the storage device if the device has SCSI UNMAP or ATA TRIM capability. Windows Server also provides an API implementation for unmapping LBAs on a storage target device. Windows Server doesn't adopt T10 SCSI WRITE SAME command sets.

UNMAP Requests from the Hyper-V Guest Operating System

During the virtual machine (VM) creation, a Hyper-V host sends an inquiry about whether the storage device that the virtual hard disk (VHD) resides on supports UNMAP or TRIM commands. When a large file is deleted from the file system of a VM guest OS, the guest OS sends a file delete request to the virtual machine’s virtual hard disk (VHD) or VHD file (or VHDX file). The VM’s VHD or VHDX file tunnels the SCSI UNMAP request to the class driver stack of the Windows Hyper-V host, as follows:

• If the VM has a VHD file, the VHD converts SCSI UNMAP or ATA TRIM commands into a Data Set Management I/O control code TRIM request, and then sends the request to the host storage device.
• If the VM has a VHDX file, the VHD file system converts SCSI UNMAP or ATA TRIM commands into file system-level trim requests, and then sends the requests to the host operating system.

Windows Hyper-V also supports IOCTL DSM TRIM calls from the guest operating system.

Windows Optimize Drives Utility

End users or system administrators can use the Optimize Drives utility to reclaim space either by creating a manual request or by optimizing the schedule configuration. If the disk drive is a thin provisioning LUN, the media type of the disk drive appears as “Thin Provisioning Drive”.

The system administrator can schedule a storage space consolidation by using the Optimize Drives utility. The utility can also notify the system administrator if the system misses three consecutive scheduled runs.

Retrieving the Slab Mapping State

In a thin provisioning LUN, all logical blocks are grouped in slabs (clusters). The OPTIMAL UNMAP GRANULARITY parameter that the storage device reports sets the slab size. All slabs are classified into Mapped, Deallocated or Anchored states. Windows Server treats both Deallocated and Anchored states as unmapped states. Windows Server provides an API implementation, or an IOCTL DSM allocation, to retrieve the LBA provisioning status from thin provisioning LUNs for the storage management operation. The application can call the IOCTL DSM allocation routine to send the SCSI command and retrieve the mapped or unmapped state of each slab in a particular range. If the LBA provisioning status returned doesn't describe the entire allocation range, the application sends another SCSI command to retrieve the provisioning status of the remaining LBA range.

The storage device doesn't need to process the entire LBA range in one return. If the partial LBA range of the original request has been returned, another command is sent to retrieve the mapping states of the remaining LBA range.