New Exchange Core Store Functionality
Applies to: Exchange Server 2010
Microsoft Exchange Server 2010 includes many improvements to the Exchange database architecture:
- Public folder reporting has been enhanced.
- Databases are no longer associated with storage groups. Storage groups have been removed.
- Investments in store schema and Extensible Storage Engine (ESE) optimizations have reduced IOPS by 70 percent.
The following sections describe these improvements in more detail.
Contents
Enhanced Reporting for Public Folders
Database Management
Store Schema Changes
New ESE Functionality
Enhanced Reporting for Public Folders
Public folder reporting has been enhanced to view user-initiated changes to any item in the public folder. You can view this information by using the Get-PublicFolderStatistics cmdlet in the Exchange Management Shell. For more information, see Exchange Management Shell.
Database Management
Databases are no longer associated with storage groups. In Exchange 2010, storage group functionality has been moved to the database.
In Exchange 2010, you can manage mailbox and public folder databases in the Organization Configuration node of the EMC. (In Exchange Server 2007, database management was performed in the Server Configuration node.)
Although public folder database management has been moved from the Server Configuration node to the Organization Configuration node with the mailbox databases, the functionality of public folder databases hasn't changed in Exchange 2010. Just like in Exchange 2007, you can't create database copies of public folder databases, and you can't add public folder databases to a database availability group (DAG). However, public folder databases can be hosted on Mailbox servers that are part of a DAG, although public folder databases won't be subject to log shipping or any other DAG features.
Database Cmdlet Changes
With the removal of storage groups in Exchange 2010, the storage group cmdlets used in Exchange 2007 were deleted and the Exchange 2010 database cmdlets now provide the functionality, as shown in the following tables.
Database cmdlets in Exchange 2010 that replace Exchange 2007 storage group cmdlets
| Exchange 2007 cmdlet | Description of functionality change in Exchange 2010 |
|---|---|
New-StorageGroup |
This cmdlet has been deleted, and configuration parameters were moved to the New-MailboxDatabase and New-PublicFolderDatabase cmdlets. |
Remove-StorageGroup |
This cmdlet has been deleted, and configuration parameters were moved to the Remove-MailboxDatabase and Remove-PublicFolderDatabase cmdlets. |
Set-StorageGroup |
This cmdlet has been deleted, and configuration parameters were moved to the Set-MailboxDatabase and the Set-PublicFolderDatabase cmdlets. |
Get-StorageGroup |
This cmdlet has been deleted, and configuration parameters were moved to the Get-MailboxDatabase and Get-PublicFolderDatabase cmdlets. |
Move-StorageGroupPath |
This cmdlet has been deleted, and configuration parameters were moved to the Move-DatabasePath cmdlet. |
Database cmdlets in Exchange 2010 that have extended functionality from Exchange 2007 cmdlets
| Exchange 2010 cmdlet | Description of extended functionality in Exchange 2010 |
|---|---|
New-MailboxDatabase New-PublicFolderDatabase |
These cmdlets have been extended with the parameters and functionality from the New-StorageGroup cmdlet. They also update the server object with a link to the new database and the database object with the hosting server name. |
Remove-MailboxDatabase Remove-PublicFolderDatabase |
These cmdlets have been extended with the parameters and functionality from the Remove-StorageGroup cmdlet. In addition, they also update the server object with the link to the new database and the database object with the hosting server name. |
Set-MailboxDatabase Set-PublicFolderDatabase |
These cmdlets have been extended with the parameters and functionality from the Set-StorageGroup cmdlet. When changing the host servers, they also update the server object with the link to the new database and the database object with the hosting server name. |
Get-MailboxDatabase Get-PublicFolderDatabase |
These cmdlets have been extended with the parameters and functionality from the Get-StorageGroup cmdlet. The Status parameter is extended to return the status information currently returned by the Get-StorageGroupCopyStatus cmdlet. |
Move-DatabasePath |
This cmdlet has been extended with the parameters and functionality from the Move-StorageGroupPath cmdlet. |
In addition to the preceding cmdlet changes, the StorageGroupCopy cmdlets have been deleted. For more information, see Managing Mailbox Database Copies.
Store Changes
In Exchange 2010, the store schema has been changed to remove the dependency of mailbox databases on the server object. In addition, the new schema has been improved to help reduce database I/O per second (IOPS) by refactoring the tables used to store information. Refactoring the tables allows higher logical contiguity and locality of reference. These changes reduce the store's reliance on the secondary indexes maintained by ESE. As a result, the store is no longer sensitive to performance issues related to the secondary indexes.
Store resilience and health has also been improved by adding several features related to detecting and correcting errors and providing alerts, such as the following:
- Mailbox quarantine on rogue mailboxes
- Transport cutoff to databases with less than 1 GB of space
- Thread time-out detection and reporting
For more information about store resilience and health, see Understanding the Exchange 2010 Store.
Core store functionality has received many changes to improve high availability features. High availability has been integrated into the core architecture of Exchange 2010 to enable organizations of all sizes and in all industry segments to economically deploy a messaging continuity service. For more information about the high availability changes in Exchange 2010, see Understanding High Availability and Site Resilience.
New ESE Functionality
Extensible Storage Engine (ESE) has been improved in Exchange 2010 to achieve the following goals:
- Larger I/O and sequential I/O to reduce IOPS
- Optimization for commodity storage
- Database management reduction
- Online defragmentation
- Online database scanning
Larger and Sequential I/O
By increasing the size of the I/O and reducing the frequency of read/writes in Exchange 2010, ESE is able to increase performance. In addition, ESE can increase performance by making the data in the database more sequential, which increases the likelihood that related data is in the same vicinity in the B-tree.
In Exchange, all data inside the database is stored in B-trees, and the B-trees are then divided into pages. In Exchange 2007 and earlier, the data stored in the B-trees isn't contiguous. In fact, previous versions of Exchange performed random read/writes to the database. This means that related data may not be in the same vicinity on the hard disk. Non-contiguous data requires more passes to read and write to the hard disk.
B-Tree Defragmentation
The B-tree defragmentation process has been improved to reduce I/O operations by maintaining contiguous data in the B-tree.
B-tree defragmentation is performed in-place (as opposed to creating a new B-tree and renaming the indexes and tables) with three new operations:
- Page move A page move consists of moving all data from one page to a newly allocated page.
- Partial left merge A partial left merge is the same as a right merge in Exchange 2007 or earlier, except that data is moved from the left page to the right page.
- Full left merge A full left merge is the same as a full right merge in Exchange 2007 or earlier.
Defragmentation has been changed from right merges to left merges to optimize performance. Data is read from or written to the hard disk from right to left. If the database is being defragmented in the same direction as the read/writes, defragmentation will conflict with the read/writes. In addition, space allocation allows the next page in an extent to be allocated, but not the previous page. Because a page move needs to allocate a new page, defragging the database from left to right is much more efficient.
The Defragmentation Manager is a new event in ESE that monitors which B-trees require defragmenting and which B-trees have already been defragmented. The Defragmentation Manager compiles a list of the B-trees in all mounted databases that should be defragmented. As fragmented B-trees are discovered, they're registered with the Defragmentation Manager, and the Defragmentation Manager will process them.
Page Size Increase to 32 KB
All data inside the database is stored in B-trees, and the B-trees are divided into pages. The page size is the minimum size for reading and writing to the database; it's also the unit size used for database caching. Reading from the disk is slower than performing operations in memory; therefore, by increasing the page size to 32 KB, ESE reduces IOPS, which increases performance by caching the larger page size in memory.
Optimize for Commodity Storage
Another of the goals of ESE in Exchange 2010 is to reduce the capital and operational costs of deploying Exchange. This can be done by reducing storage costs and optimizing for commodity storage using JBOD and SATA class hard disks.
Disk subsystems are more efficient at handling fewer but larger I/O. In Exchange 2010 or earlier, the page size is the minimum read/write size and the minimum size for database caching. Coalescing I/Os refers to the process of combining database page operations into a single I/O operation, thereby producing fewer and bigger I/O operations.
Increasing the average database I/O sizes via coalescing I/Os has the following benefits:
- Increased disk use efficiency Disks are more efficient at processing large I/Os. The more efficiently the disk is utilized, the more mailboxes can be hosted on that disk.
- Increased cache warming rate Cache warming is a process that helps reduce the execution times by preloading the initial queries that were executed against a database the last time the database was started. After a server restart, failover, or switchover, the larger I/O allows ESE to increase the rate at which the cache is warmed.
Database Maintenance
One of the goals of ESE in Exchange 2010 is to reduce the cost of maintaining and managing a database. Database maintenance is comprised of several tasks that manage and keep the integrity of your mailbox database.
Database maintenance is divided into the following:
- Store mailbox maintenance
- ESE database maintenance
In Exchange 2007, ESE database maintenance was disk-intensive. In Exchange 2010, improvements have been made to increase performance. In Exchange 2010, on large or very heavy profile servers, the store mailbox maintenance task only lasts approximately 45 minutes, while ESE database maintenance usually took from six to eight hours per night to complete on large Exchange 2007 databases (2 GB quotas).
In Exchange 2010, improvements have been made to support both large mailboxes as well as to support JBOD storage and storage without the use of RAID.
Note
All Exchange Store-focused online database maintenance functions like recovery item cleanup are the same in Exchange 2010 as they were in Exchange 2007. Only ESE functions, online defragmentation, and database checksumming have changed.
Online Defragmentation
In Exchange 2010, the architecture for online defragmentation has changed. Online defragmentation was moved out of the Mailbox database maintenance process. Online defragmentation now runs in the background 24×7.
You don't need to configure any settings for this feature. Exchange monitors the database as it's being used, and small changes are made over time to keep it defragged for space and contiguity. If the database analyzes a range of pages and finds that the pages aren't as sequential as they should be, it starts an async thread to defragment that section of the B-tree/table. Online defragmentation is also throttled so it doesn't have a negative impact on client performance.
Use the ESE performance counter set MSExchange Database ==> Defragmentation Tasks to see the tasks that are performed. For more information, see How to Enable Extended ESE Performance Counters.
Online Database Scanning
Online database scanning (also known as database checksumming) has also changed. In Exchange 2007 Service Pack 1 (SP1), you had the option to use half of your online defragmentation time for this database scanning process (to ensure Exchange read every page from your database in a specific period of time to detect any corruptions). This I/O is 100 percent sequential (thus easy on the disk) and on most systems equated to a checksumming rate of about 5 megabytes (MB)/sec.
In Exchange 2010, online database scanning simply checksums the database and performs post-Exchange 2010 Store crash operations. Space leaks can occur due to crashes, and online database scanning finds and recovers this lost space.
The system in Exchange 2010 is designed with the expectation that every database is fully scanned once every three days. A warning event is fired if the databases aren't scanned. In Exchange 2010, there are now two modes to run online database scanning on active database copies:
- Run as the last task in the scheduled Mailbox Database Maintenance process. You can configure how long it runs by changing the Mailbox Database Maintenance schedule. This option works well for smaller databases that are less than 1 terabyte (TB) in size.
- Run in the background 24×7, the default behavior. This option works well for larger databases, up to 2 TB, where you need more time to checksum the databases. Exchange scans the database no more than once per day and fires a warning event if it can't finish scanning in a seven-day period.
You can configure online database scanning to not run 24x7 by clearing the check box (if you don't want the I/O overhead during peak period). When you do this, the last part of the online database scanning window is used to do the database checksumming and lost space recovery.
For more information about configuring database maintenance, see Maintain Mailbox Databases.