Understanding Storage Configuration
Applies to: Exchange Server 2010 SP3, Exchange Server 2010 SP2
Understanding storage options and requirements for the Mailbox server role in Microsoft Exchange Server 2010 is an important part of your Mailbox server storage design solution. For additional information about other key aspects of the design process, see Mailbox Server Storage Design.
Contents
Storage Architectures
Physical Disk Types
Best Practices for Supported Storage Configurations
Storage Architectures
The following table describes supported storage architectures, and provides best practice guidance for each type of storage architecture, where appropriate.
Supported storage architectures
| Storage architecture | Description | Best practice |
|---|---|---|
Direct-attached storage (DAS) |
DAS is a digital storage system directly attached to a server or workstation, without a storage network in between. For example, DAS transports include Serial Attached Small Computer System Interface (SCSI) and Serial Attached Advanced Technology Attachment (ATA). |
Not available. |
Storage area network (SAN): Internet Small Computer System Interface (iSCSI) |
SAN is an architecture to attach remote computer storage devices (such as disk arrays and tape libraries) to servers in such a way that the devices appear as locally attached to the operating system (for example, block storage). iSCSI SANs encapsulate SCSI commands within IP packets and use standard networking infrastructure as the storage transport (for example, Ethernet). |
Don't share physical disks backing up Exchange data with other applications. Use dedicated storage networks. Use multiple network paths for stand-alone configurations. |
SAN: Fibre Channel |
Fibre Channel SANs encapsulate SCSI commands within Fibre Channel packets and generally utilize specialized Fibre Channel networks as the storage transport. |
Don't share physical disks backing up Exchange data with other applications. Use multiple Fibre Channel network paths for stand-alone configurations. Follow storage vendor's best practices for tuning Fibre Channel host bus adapters (HBAs), for example, Queue Depth and Queue Target. |
Return to top
Physical Disk Types
The following table provides a list of supported physical disk types and provides best practice guidance for each physical disk type where appropriate.
Supported physical disk types
| Physical disk type | Description | Supported or best practice |
|---|---|---|
Serial ATA (SATA) |
SATA is a serial interface for ATA and integrated device electronics (IDE) disks. SATA disks are available in a variety of form factors, speeds, and capacities. In general, choose SATA disks for Exchange 2010 mailbox storage when you have the following design requirements:
|
Supported: 512-byte sector disks for Windows Server 2008 and Windows Server 2008 R2. In addition, 512e disks are supported for Windows Server 2008 R2 with the following:
Support requires that all copies of a database reside on the same physical disk type. For example, it is not a supported configuration to host one copy of a given database on a 512-byte sector disk and another copy of that same database on a 512e disk. Also be aware that 4-kilobyte (KB) sector disks are not supported for any version of Microsoft Exchange and 512e disks are not supported for any version of Exchange prior to Exchange Server 2010 SP1. Best practice: Consider enterprise class SATA disks, which generally have better heat, vibration, and reliability characteristics. |
Serial Attached SCSI |
Serial Attached SCSI is a serial interface for SCSI disks. Serial Attached SCSI disks are available in a variety of form factors, speeds, and capacities. In general, choose Serial Attached SCSI disks for Exchange 2010 mailbox storage when you have the following design requirements:
|
Supported: 512-byte sector disks for Windows Server 2008 and Windows Server 2008 R2. In addition, 512e disks are supported for Windows Server 2008 R2 with the following:
Support requires that all copies of a database reside on the same physical disk type. For example, it is not a supported configuration to host one copy of a given database on a 512-byte sector disk and another copy of that same database on a 512e disk. Also be aware that 4-kilobyte (KB) sector disks are not supported for any version of Microsoft Exchange and 512e disks are not supported for any version of Exchange prior to Exchange Server 2010 SP1. Best practice: Physical disk-write caching must be disabled when used without a UPS. |
Fibre Channel |
Fibre Channel is an electrical interface used to connect disks to Fibre Channel-based SANs. Fibre Channel disks are available in a variety of speeds and capacities. In general, choose Fibre Channel disks for Exchange 2010 mailbox storage when you have the following design requirements:
|
Supported: 512-byte sector disks for Windows Server 2008 and Windows Server 2008 R2. In addition, 512e disks are supported for Windows Server 2008 R2 with the following:
Support requires that all copies of a database reside on the same physical disk type. For example, it is not a supported configuration to host one copy of a given database on a 512-byte sector disk and another copy of that same database on a 512e disk. Also be aware that 4-kilobyte (KB) sector disks are not supported for any version of Microsoft Exchange and 512e disks are not supported for any version of Exchange prior to Exchange Server 2010 SP1. Best practice: Physical disk-write caching must be disabled when used without a UPS. |
Solid-state drive (SSD) (flash disk) |
An SSD is a data storage device that uses solid-state memory to store persistent data. An SSD emulates a hard disk drive interface. SSD disks are available in a variety of speeds (different I/O performance capabilities) and capacities. In general, choose SSD disks for Exchange 2010 mailbox storage when you have the following design requirements:
|
Supported: 512-byte sector disks for Windows Server 2008 and Windows Server 2008 R2. In addition, 512e disks are supported for Windows Server 2008 R2 with the following:
Support requires that all copies of a database reside on the same physical disk type. For example, it is not a supported configuration to host one copy of a given database on a 512-byte sector disk and another copy of that same database on a 512e disk. Also be aware that 4-kilobyte (KB) sector disks are not supported for any version of Microsoft Exchange and 512e disks are not supported for any version of Exchange prior to Exchange Server 2010 SP1. Best practice: Physical disk-write caching must be disabled when used without a UPS. In general, Exchange 2010 Mailbox servers don't require the performance characteristics of SSD storage. |
Factors to Consider When Choosing Disk Types
There are several trade-offs when choosing disk types for Exchange 2010 storage. The correct disk is one that balances performance (both sequential and random) with capacity, reliability, power utilization, and capital cost. The following table of supported physical disk types provides information to help you when considering these factors.
Factors in disk type choice
| Disk speed (RPM) | Disk form factor | Interface or transport | Capacity | Random I/O performance | Sequential I/O performance | Power utilization |
|---|---|---|---|---|---|---|
5,400 |
2.5-inch |
SATA |
Average |
Poor |
Poor |
Excellent |
5,400 |
3.5-inch |
SATA |
Excellent |
Poor |
Poor |
Above average |
7,200 |
2.5-inch |
SATA |
Average |
Average |
Average |
Excellent |
7,200 |
2.5-inch |
Serial Attached SCSI |
Average |
Average |
Above average |
Excellent |
7,200 |
3.5-inch |
SATA |
Excellent |
Average |
Above average |
Above average |
7,200 |
3.5-inch |
Serial Attached SCSI |
Excellent |
Average |
Above average |
Above average |
7,200 |
3.5-inch |
Fibre Channel |
Excellent |
Average |
Above average |
Average |
10,000 |
2.5-inch |
Serial Attached SCSI |
Below average |
Excellent |
Above average |
Above average |
10,000 |
3.5-inch |
SATA |
Average |
Average |
Above average |
Above average |
10,000 |
3.5-inch |
Serial Attached SCSI |
Average |
Above average |
Above average |
Below average |
10,000 |
3.5-inch |
Fibre Channel |
Average |
Above average |
Above average |
Below average |
15,000 |
2.5-inch |
Serial Attached SCSI |
Poor |
Excellent |
Excellent |
Average |
15,000 |
3.5-inch |
Serial Attached SCSI |
Average |
Excellent |
Excellent |
Below average |
15,000 |
3.5-inch |
Fibre Channel |
Average |
Excellent |
Excellent |
Poor |
SSD: enterprise class |
Not applicable |
SATA, Serial Attached SCSI, Fibre Channel |
Poor |
Excellent |
Excellent |
Excellent |
Return to top
Best Practices for Supported Storage Configurations
This section provides best practice information about supported disk and array controller configurations.
Redundant Array of Independent Disks (RAID) is often used to both improve the performance characteristics of individual disks (by striping data across several disks) as well as to provide protection from individual disk failures. With the advancements in Exchange 2010 high availability, RAID is no longer a required component for Exchange 2010 storage design. However, RAID is still an essential piece to Exchange 2010 storage design for stand-alone servers as well as high availability solutions that require either additional performance or greater storage reliability. The following table provides guidance for the common RAID types that can be used with the Exchange 2010 Mailbox server.
Supported data types for the Exchange 2010 Mailbox server role
| Data type | Stand-alone: supported or best practice | High availability: supported or best practice |
|---|---|---|
OS, system, or pagefile volume |
Supported: All RAID types. Best practice: RAID1/10. Use a dedicated array group; don't host both system LUN and data LUNs on the same array group. |
Supported: All RAID types. Best practice: RAID1/10. Use a dedicated array group; don't host both system LUN and data LUNs on the same array group. |
Exchange mailbox database (.edb) file volume |
Supported: All RAID types. Best practice: 5,400 or 7,200 disks = RAID1/10 only. RAID5* = Maximum of 7 disks per array group and array controller high priority scrubbing and surface scanning enabled. RAID6* = High priority scrubbing and surface scanning enabled. |
Supported: All RAID types. Just a bunch of disks (JBOD) (not RAID) (three or more database copies). Best practice: 5,400 or 7,200 disks = RAID1/10 only or JBOD. When lagged, database copies should have two or more lagged copies, or lagged copies should be protected with RAID. RAID5* = Maximum of 7 disks per array group and array controller high priority scrubbing and surface scanning enabled. RAID6* = High priority scrubbing and surface scanning enabled. |
Exchange mailbox database log volume |
Supported: All RAID types. Best practice: RAID1/10. |
Supported: All RAID types. JBOD (not RAID) (three or more database copies). Best practice: RAID1/10. When lagged, database copies should have two or more lagged copies, or lagged copies should be protected with RAID. |
*Includes RAID variations such as RAID50 or RAID51 for RAID5
The following table provides guidance about storage array configurations for Exchange 2010.
Supported RAID types for the Exchange 2010 Mailbox server role
| RAID type | Description | Supported or best practice |
|---|---|---|
Disk array RAID stripe size (KB) |
The stripe size is the per disk unit of data distribution within a RAID set. Stripe size is also referred to as block size. |
Best practice: 256 KB or greater. Follow storage vendor best practices. |
Storage array cache settings |
The cache settings are provided by a battery-backed caching array controller. |
Best practice: 75 percent write cache and 25 percent read cache (battery-backed cache). Follow storage vendor best practices. |
Physical disk write caching |
The settings for the cache are on each individual disk. |
Supported: Physical disk write caching must be disabled when used without a UPS. |
The following table provides guidance about database and log file choices.
Database and log file choices for the Exchange 2010 Mailbox server role
| Database and log file options | Description | Stand-alone: supported or best practice | High availability: supported or best practice |
|---|---|---|---|
File placement: database per log isolation |
Database per log isolation refers to placing the database file and logs from the same mailbox database onto different volumes backed by different physical disks. |
Best practice: For recoverability, move database (.edb) file and logs from the same database to different volumes backed by different physical disks. |
Supported: Isolation of logs and databases isn't required. |
File placement: database files per volume |
Database files per volume refers to how you distribute database files within or across disk volumes. |
Best practice: Based on your backup methodology. |
Supported: When using JBOD, divide a single disk into two volumes (one for database; one for log stream). |
File placement: log streams per volume |
Log streams per volume refers to how you distribute database log files within or across disk volumes. |
Best practice: Based on your backup methodology. |
Supported: When using JBOD, divide a single disk into two volumes (one for database; one for log stream). Best practice: When using JBOD, single database per log per volume. |
Database size |
Database size refers to the disk database (.edb) file size. |
Supported: Approximately 16 terabytes. Best practice:
|
Supported: Approximately 16 terabytes. Best practice:
|
Log truncation method |
Log truncation method is the process for truncating and deleting old database log files. There are two mechanisms:
|
Best practice:
|
Best practice:
|
The following table provides guidance about Windows disk types.
Windows disk types for the Exchange 2010 Mailbox server role
| Windows disk type | Description | Stand-alone: supported or best practice | High availability: supported or best practice |
|---|---|---|---|
Basic disk |
A disk initialized for basic storage is called a basic disk. A basic disk contains basic volumes, such as primary partitions, extended partitions, and logical drives. |
Supported. Best practice: Use basic disks. |
Supported. Best practice: Use basic disks. |
Dynamic disk |
A disk initialized for dynamic storage is called a dynamic disk. A dynamic disk contains dynamic volumes, such as simple volumes, spanned volumes, striped volumes, mirrored volumes, and RAID-5 volumes. |
Supported. |
Supported. |
The following table provides guidance on volume configurations.
Volume configurations for the Exchange 2010 Mailbox server role
| Volume configuration | Description | Stand-alone: supported or best practice | High availability: supported or best practice |
|---|---|---|---|
GUID partition table (GPT) |
GPT is a disk architecture that expands on the older master boot record (MBR) partitioning scheme. The maximum NTFS formatted partition size is 256 terabytes. |
Supported. Best practice: Use GPT partitions. |
Supported. Best practice: Use GPT partitions. |
MBR |
An MBR, or partition sector, is the 512-byte boot sector that is the first sector (LBA Sector 0) of a partitioned data storage device such as a hard disk. The maximum NTFS formatted partition size is 2 terabytes. |
Supported. |
Supported. |
Partition alignment |
Partition alignment refers to aligning partitions on sector boundaries for optimal performance. |
Supported: The Windows Server 2008 default is 1 megabyte (MB). |
Supported: The Windows Server 2008 default is 1 MB. |
Volume path |
Volume path refers to how a volume is accessed. |
Supported: Drive letter or mount point. Best practice: Mount point host volume must be RAID enabled. |
Supported: Drive letter or mount point. Best practice: Mount point host volume must be RAID-enabled. |
File system |
File system is a method for storing and organizing computer files and the data they contain to make it easy to find and access the files. |
Supported: NTFS support only. |
Supported: NTFS support only. |
NTFS defragmentation |
NTFS defragmentation is a process that reduces the amount of fragmentation in Windows file systems. It does this by physically organizing the contents of the disk to store the pieces of each file close together and contiguously. |
Supported. Best practice: Not required and not recommended. |
Supported. Best practice: Not required and not recommended. |
NTFS allocation unit size |
NTFS allocation unit size represents the smallest amount of disk space that can be allocated to hold a file. |
Supported: All allocation unit sizes. Best practice: 64 KB for both .edb and log file volumes. |
Supported: All allocation unit sizes. Best practice: 64 KB for both .edb and log file volumes. |
NTFS compression |
NTFS compression is the process of reducing the actual size of a file stored on the hard disk. |
Supported: Not supported for Exchange database or log files. |
Supported: Not supported for Exchange database or log files. |
NTFS Encrypting File System (EFS) |
EFS enables users to encrypt individual files, folders, or entire data drives. Because EFS provides strong encryption through industry-standard algorithms and public key cryptography, encrypted files are confidential even if an attacker bypasses system security. |
Supported: Not supported for Exchange database or log files. |
Not supported for Exchange database or log files. |
Windows BitLocker (volume encryption) |
Windows BitLocker is a data protection feature in Windows Server 2008. BitLocker protects against data theft or exposure on computers that are lost or stolen, and it offers more secure data deletion when computers are decommissioned. |
Supported: All Exchange database and log files. |
Supported: All Exchange database and log files. Windows failover clusters require Windows Server 2008 R2 or Windows Server 2008 R2 SP1 and the following hotfix: You cannot enable BitLocker on a disk volume in Windows Server 2008 R2 if the computer is a failover cluster node. Exchange volumes with Bitlocker enabled are not supported on Windows failover clusters running earlier versions of Windows. For more information about Windows 7 BitLocker encryption, see BitLocker Drive Encryption in Windows 7: Frequently Asked Questions. |
Return to top
Windows Disk Timeouts
Starting with Exchange 2010 Service Pack 1 (SP1), intelligence is included to deal with hung I/O. Before Exchange 2010, Exchange reported slow I/O in the event log, but does not take any other action. Exchange 2010 SP1 will actively fail (bugcheck) the server if the hung I/O is affecting active databases on a DAG node.
The new recovery logic in Exchange 2010 SP1 leverages the built-in Windows bugcheck behavior when certain conditions occur. Specifically, when hung IO occurs. The Extensible Storage Engine (ESE) has been updated to detect hung IO and to take corrective action to automatically recover the server.
ESE maintains an IO watchdog thread that detects when an IO has been outstanding for a specific period of time. By default, if an IO for a database is outstanding for more than one minute, ESE logs an event. If a database has an IO that has been outstanding for more than four minutes, ESE logs a specific failure event, if it is possible to do so.
ESE Event 507, 508, 509, or 510 may or may not be logged, depending on the nature of the hung IO. If the nature of the problem is such that the OS volume is affected or the ability to write to the event log is affected, the events are not logged. If the events are logged, the Microsoft Exchange Replication service (MSExchangeRepl.exe) detects that condition, and intentionally cause a bugcheck of Windows by terminating the wininit.exe process. The following table describes the recovery logic behavior in Exchange 2010 SP1 and earlier versions.
| Exchange Version | I/O Type | I/O Time | Behavior |
|---|---|---|---|
Exchange Server 2003 |
Completed |
>60 seconds |
Write to Event Log |
Exchange Server 2007 |
Completed |
>60 seconds |
Write to Event Log |
Exchange 2010 RTM |
Completed |
>60 seconds |
Write to Event Log ESE performs clean-page overwrite on pages affected by slow I/O |
Exchange 2010 SP1 |
In Flight |
>60 seconds |
Write to Event Log |
Exchange 2010 SP1 |
In Flight |
>4 minutes |
Terminate wininit.exe process and bugcheck the server |
Exchange Server 2010 SP1 |
Completed |
>30 seconds |
Write to event log ESE performs clean-page overwrite on pages affected by slow I/O |
Note
In the I/O Type column of the table, In Flight describes a slow I/O operation that has not yet successfully finished. and Completed describes a slow I/O operation that took more than 30 seconds to finish. The concept of detecting slow I/O in-flight operations is new in Exchange 2010. In earlier versions of Microsoft Exchange, the program reported only after the I/O had finished.
We recommend that you do not change the new recovery logic behavior in Exchange 2010 SP1. However, if you must change the new behavior, see New High Availability and Site Resilience Functionality in Exchange 2010 SP1 for more information about how to do this.
The following table outlines the recommended guidance for setting the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Disk\TimeOutValue registry subkey for servers that are running the Exchange 2010 Mailbox role.
| Scenario | Recommendation |
|---|---|
Direct-Attached Storage |
Reduce Windows disk TimeOutValue to 20 seconds Refer to hardware manufacturer’s guidance Hardware manufacturer’s guidance takes priority in the event of a clash |
SAN-Attached RAID Storage |
Reduce Windows disk TimeOutValue to 20 seconds Refer to hardware manufacturer’s guidance Hardware manufacturer’s guidance takes priority in the event of a clash |
JBOD Storage |
Increase Windows disk TimeOutValue to 180 seconds Refer to hardware manufacturer’s guidance Hardware manufacturer’s guidance takes priority in the event of a clash |
For more information, see the Exchange Server Team Blog article Windows Disk Timeouts and Exchange Server 2010.
© 2010 Microsoft Corporation. All rights reserved.