Exchange Q&AA Second Look at Exchange 2010 Features and Improvements

Henrik Walther

In this installment, I’m continuing where I left off in my last column—that is, answering even more questions related to new and exciting Exchange 2010 features. But I also have room for a couple of Exchange Server 2007 questions.

Q We are deploying a multisite Exchange 2007 SP1 cluster using Cluster Continuous Replication (CCR). The two cluster nodes will be located in separate datacenters. Exchange runs on Windows Server 2008 SP2 and we plan to have the public and private interfaces located in different subnets in each datacenter. As you know, this means we must use routing between the cluster nodes.

We have no problem configuring the public interface according to the instructions in the CCR section on Microsoft TechNet. But when we configure the default gateway on the private interface, we receive the warning message shown in Figure 1.

Figure 1 Specifying multiple default gateways on a Windows 2008 Server.

Based on this warning message, we suspect things will not work properly if we specify multiple default gateways on each node in our multisite CCR cluster. This leads us to our question: How should we configure the private network interface in this type of scenario?

A I'm glad you asked this question before proceeding, because specifying multiple default gateways on a multisite CCR cluster will cause major issues. The proper configuration requires persistent, static routes for each private interface.

To get started, make sure the public interface is listed first on the connection order list under Advanced Settings in the Network Connections control panel. Next, make sure you have specified a default gateway on the public network interface for each cluster node.

Finally, configure routes on the private interfaces so that all traffic that doesn't match the route created will use the default gateway of the public interface. Let's imagine the node 1 private interface is on subnet 192.168.100.x/24 with a network gateway at IP address 192.168.100.1. The node 2 private network interface is on subnet 192.168.200.x/24 with a network gateway at IP address 192.168.200.1. In this case you would need to create this route on node 1: ROUTE ADD 192.168.200.0 MASK 255.255.255.0 192.168.100.1 –P, and the following route on node 2: ROUTE ADD 192.168.100.0 MASK 255.255.255.0 192.168.200.1 –P.

The –P parameter specifies that the created routes are persistent and won't be cleared after a reboot.

This configuration will ensure proper networking for each interface in the cluster nodes. For even more details on how to configure the private interface when using routing between the networks, check out the blog post by my good friend Tim McMichael.

On a final note, I also recommend you configure the private network as a mixed network so that the Enable-ContinuousReplicationHostName cmdlet can be used to direct replication activity over the redundant network. For more information about this cmdlet, see How to Enable Redundant Cluster Networks for Log Shipping and Seeding on Windows Server 2003.

Q I have a question regarding the new Database Availability Group (DAG) feature in Exchange Server 2010. More specifically, my question relates to log file copying and seeding between active and passive databases. Will Exchange Server 2010 offer changes or improvements in the way log file copying and seeding occurs with Local Continuous Replication (LCR), Cluster Continuous Replication (CCR) and Standby Continuous Replication (SCR) in Exchange Server 2007?

A That is a great question. Although the asynchronous replication technology used in Exchange 2007 works quite well, that doesn't mean it can't be improved, right? I'm glad to inform you that the Exchange Product Group has made several interesting changes and improvements to the asynchronous replication technology with Exchange 2010.

In Exchange 2007, the Microsoft Exchange Replication Service copies log files to the passive database copy (LCR), passive cluster node (CCR) or SCR target over Server Message Block (SMB), which means you need to open port 445 in any firewalls between the CCR cluster nodes (typically when deploying multisite CCR clusters) and/or SCR source and targets. Those of you who work for or with a large enterprise organization know that convincing network administrators to open port 445/TCP between two datacenters a far from a trivial exercise. With the Exchange 2010 DAG feature, the asynchronous replication technology no longer relies on SMB. Exchange 2010 uses TCP/IP for log file copying and seeding and, even better, it provides the option of specifying which port you want to use for log file replication. By default, DAG uses port 64327, but you can specify another port if required. For this, use the following command:

Set-DatabaseAvailabilityGroup -identity <DAG name> -ReplicationPort <port number>

In addition, the Exchange 2010 DAG feature supports the use of encryption whereas log files in Exchange 2007 are copied over an unencrypted channel unless IPsec has been configured. More specifically, DAG leverages the encryption capabilities of Windows Server 2008—that is, DAG uses Kerberos authentication between each Mailbox server member of the respective DAG. Network encryption is a property of the DAG itself, not the DAG network. Settings for a DAG's network encryption property are: Disabled (network encryption not in use), Enabled (network encryption enabled for seeding and replication on all networks existing in a DAG), InterSubnetOnly (the default setting meaning network encryption in use on DAG networks on the same subnet), and SeedOnly (network encryption in use for seeding on all networks in a DAG). You can enable network encryption using the Set-DatabaseAvailabilityGroupcmdlet. For instance, if you wanted to enable encryption for log copying and seeding, you would execute the command:

Set-DatabaseAvailabilityGroup -identity <DAG name> -NetworkEncryption Enabled.

Finally, with Exchange 2010 DAGs you can enable compression for seeding and replication over one or more networks in a DAG. This is a property of the DAG itself, not a DAG network. The default setting is InterSubnetOnly and has the same settings available as those of the network encryption property. To enable network compression for log file copying and seeding on all networks in a DAG, use the command: Set-DatabaseAvailabilityGroup –Identity <DAG name> -NetworkCompression Enabled. To find the status of the port, encryption and compression settings for a DAG, use the Get-DatabaseAvailabilityGroup –status command as shown in Figure 2.

Figure 2 Database Availability Group network encryption, compression and replication port settings.

Q In your previous Exchange Queue & A column, you talked about a significant I/O reduction in Exchange 2010 compared to Exchange 2003 and 2007. Can you explain the specific changes and improvements the Exchange Product Group made to achieve such optimization for storage performance in Exchange 2010?

A As you mentioned, we saw a significant reduction in I/Os per second in the move from Exchange 2003 to Exchange 2007. Reductions of up to 70 percent were not uncommon. We can attribute this primarily to the use of a 64-bit architecture for Exchange 2007. As such, Exchange 2007 can access more memory and thereby use larger memory caches than its predecessor. The more data Exchange can retrieve directly from the virtual memory address space, the fewerI/Os needed against the disks in the underlying storage subsystem.

This provides much more efficient use of the existing storage system (typically an expensive Storage Area Network) or a great excuse to move to less expensive direct-attached storage. The reduced I/O requirements enable hosting of many more mailboxes (5,000-plus) per Exchange 2007 Mailbox server than was possible with Exchange 2003. In order to avoid virtual memory fragmentation, Exchange 2003 was typically limited to 4,000 mailboxes per Mailbox server. (Yes, I know this depended on type of servers, storage user profiles and Exchange infrastructure.)

The Extensible Storage Engine (ESE) has been the underlying database technology in Exchange since the first version—Exchange 4.0, released in June 1996. Until Exchange 2007 SP1, the Exchange Product Group made great efforts in tweaking and optimizing the ESE for better performance.For storage in Exchange 2010, the Exchange Product Group focused on delivering large (+10GB), fast mailboxes while taking advantage of cheap storage. Because of the ESE changes made in this version, you now have the option of using such low-performance disks as the desktop-like SATA disks (aka SATA or Tier 2 disks). Yes, I am talking about 7200 SATA disks, just like the ones in your workstation! If you use the Database Availability Group feature for high availability (three or more database copies), you can even use, say, a single 7200RPM disk storing both a database and the transaction logs instead of expensive RAID configurations.

Achieving such improvements meant changing the store schema significantly. Essentially, the Exchange Product Group wanted to move away from many, random, small I/Os to fewer, sequential, large I/Os. Moving from random to sequential I/Os required dramatic changes to the store table architecture.

In Exchange 2007 and earlier, each database had a mailbox table (stored all mailboxes in the database), folders table (stored mailbox folders for all mailboxes in the database), message table (stored messages), attachment table (stored attachments for all mailboxes in the database), and message/folder table (stored folder views for all mailboxes in the database).In this architecture, which has not changed much since Exchange 4.0, a lot of random I/Os had to perform against the database. One of the benefits with this architecture was single-instance storage (SIS) in which only one copy of a message was kept—a big advantage back when relatively small disks were par for the course. But today, with 500GB SAS and 2TB SATA disks at our disposal, this architecture no longer makes sense.

In Exchange 2010, the store schema has changed so that all data in a mailbox is stored in tables close to each other in the database. Actually, each mailbox has its own folder, message header, body and view table. So, SIS no longer exists in Exchange databases. As a side effect of removing SIS from Exchange, a database might bloat by approximately 20 percent. To solve this problem, the Exchange Product Group compressed the databases (more specifically, the message headers and text or HTML bodies). By giving each mailbox its own set of tables, I/Os performed against a database are mostly sequential.

Other interesting changes include: the database space is allocated in a contiguous manner; database contiguity is maintained over time; the database page size has been increased from 8KBto 32KB; and asynchronous read capability has been improved. The Exchange Product Groupis also working on increasing the cache effectiveness by changing the checkpoint depth to 100MB for high availability configurations, using cache compression, and database cache priority.

As a result of all the changes in Exchange 2010, you can expect up to 70 percent reduction in I/O compared to Exchange 2007.

I've only scratched the surface when it comes to storage optimizations in Exchange 2010. If you want some great insight along with all the gory details on storage improvements in Exchange 2010, I highly recommend you grab a fresh cup of coffee and watch the recording of the Tech·Ed 2009 session on Exchange 2010 Storage presented by Matt Gossage, the Exchange Product Group team member responsible for ESE database. He does a great job of explaining this complex Level 300 topic in plain English. Watch it at online (you don't need to have attended Tech·Ed to watch it).

Q We have several Exchange 2007 Cluster Continuous Replication (CCR)-based Mailbox servers deployed within our organization. The private network on all nodes has File and Printer Sharing for Microsoft Networks disabled (see Figure 3). We're sure this was recommended back when we deployed Exchange 2007.

Figure 3 File and Printer Sharing for Microsoft Networks disabled.

But this differs from the guidance provided with Exchange 2007 SP1 documentation. Now the recommendation is to enable File and Printer Sharing for Microsoft Networks on the private network interface just like it is on the public network interface. Can you explain why this guidance changed after Exchange 2007 SP1 released?

A You're correct. Prior to the release of Exchange Server 2007 SP1, disabling File and Printer Sharing for Microsoft Networks on the private network interface was recommended. But in Exchange 2007 SP1, a new feature allows you to replicate log files over multiple network interfaces, thereby making the networks redundant, plus reducing the amount of data traveling over the public network interface. In the release to manufacturing (RTM) version of Exchange 2007, all log file copying and seeding occurred over the public network interface.

On a related note, if you use Windows Server 2008-based machines, make sure to enable NetBIOS on all network interfaces that you plan to use for log file copying and seeding. If NetBIOS is disabled, the Enable-ContinuousReplicationHostNames command will fail.

The Enable-ContinuousReplicationHostNames cmdlet lets you specify which network interfaces should be used for log file copying and seeding. But bear in mind that you must change the private network to a mixed network within the Windows Failover Cluster console in order to utilize it for log file copying and seeding.

See the Exchange 2007 SP1 documentation for more information.

Q In Exchange 2007, does Microsoft have a recommendation on how to stop mailflow to and from an Exchange 2007 Mailbox server, as in the case of a virus or similar outbreak? We would like users to be able to connect to their mailboxes but not be able to send and receive e-mail messages in such circumstances.

A You can do this by stopping the Microsoft Exchange Transport service on the Exchange 2007 Hub Transport servers.

If you want to have all queued e-mail messages delivered without allowing new messages into the queue, simply pause the Microsoft Exchange Transport service.

Note that doing this at the Mailbox server level requires that you stop the Microsoft Exchange Mail Submission service on the involved Exchange 2007 Mailbox servers.

Q I have taken a close look at the new archive mailbox feature in Exchange Server 2010 and have a couple of questions. I noticed that once you enable the archiving mailbox feature for a user mailbox, it's accessible and works as expected via Outlook Web Access 2010 and Outlook 2010, but not via Outlook 2003 or 2007. Will we not be able to make use of the new archive mailbox feature before we upgrade to Outlook 2010?

Also, we want to move the archive mailbox for a respective user to another mailbox database. But as far as I can see, this is not possible. If you can't move the archive mailbox to another mailbox database, what's the benefit of the archive mailbox?

A In regard to your first question, what you see is expected behavior (even when we reach Exchange 2010 RTM). You will only be able to access an archive (aka alternate) mailbox via Outlook Web Access 2010 or Outlook 2010. Accessing this mailbox via legacy Outlook 2003 or 2007 clients is neither supported nor possible..

To address your second question, you can store the archive mailbox only in the same mailbox database as the one in which you've stored your primary mailbox. But because you can use Tier 2 disks (aka SATA desktop-like disks) for your mailbox databases and logs in Exchange 2010, moving the archive mailbox to another storage subsystem doesn't really make sense. With that said, here are some benefits of the archive mailbox feature:

You can provide larger mailboxes to the end users without impacting performance of critical path folders in the primary mailbox (primary mailbox = hot data, archive mailbox = cold data) even though the mailbox databases are stored on SATA/Tier 2 disks. You can provide large mailboxes (+10GB) without synching all that data to the user's .OST file when running in cached mode (an archive mailbox is accessible only in online mode).

The archive mailbox is a replacement for .PST files. As you already know, the archive mailbox is not only accessible via Outlook 2010, but also Outlook Web Access 2010. This means you no longer need to worry about backing up your .PST files and so on.

Henrik Walther is a Microsoft Certified Master: Exchange 2007 and Exchange MVP with more than 15 years of experience in the IT business. He works as a technology architect for Trifork Infrastructure Consulting (a Microsoft Gold Certified Partner based in Denmark) and as a technical writer for Biblioso Corp. (a U.S.-based company that specializes in managed documentation and localization services).