Understanding Multiple Server Role Configurations in Capacity Planning
Applies to: Exchange Server 2010 SP3, Exchange Server 2010 SP2
Several trends in server hardware apply to the Microsoft Exchange Server 2010 timeframe. One trend is a significant increase in processor performance and an increasing number of processor cores supported on a physical processor. This means that deploying a single Exchange server role on a standard commodity server with multi-core processors might leave a large portion of available CPU underutilized. Some customers expect server virtualization to more effectively utilize server CPU resources. Other customers want to combine Exchange server roles on the same physical server. Both are valid solutions.
Another trend is the availability of server models with multi-core processors and 10 to 16 internal disks. If you consider the number of mailboxes that can be supported by the input/output transactions per second (IOPS) provided by 10 to 16 disks, the Mailbox server role by itself generally won't utilize more than half of the available CPU resources. Adding the Client Access server role and the Hub Transport server role to this server will more effectively utilize the capacity of the server.
You can use the information in this topic as guidance for when to deploy multiple-role server configurations and how to correctly plan for multiple-role server configurations. An example illustrates the server sizing process for multiple-role servers.
Contents
Why Multiple-Role Configurations are Recommended
When Multiple-Role Configurations Are Recommended
When Multiple-Role Configurations Aren't Recommended
Hardware Recommendations for Multiple-Role Servers
Deploying a Multi-Role Server in a DAG
Example of Sizing for an Exchange 2010 Multiple-Role Scenario
Why Multiple-Role Configurations are Recommended
First, and foremost, the hardware you can procure today has processors that are extremely fast, yielding 5,000-6,000 megacycles when compared to our baseline processor configuration. That configuration consists of 2 x 4 core Intel Xeon x5470 3.33-GHz processors. (You can read more about our baseline processor configuration in the section "Example of Capacity Planning for a Mailbox Server" in Mailbox Server Processor Capacity Planning.) If you were to replace the processor architecture in your environment with processors on the market today, while keeping all other environmental factors the same, you would see a significant decrease in processor utilization.
To effectively lower the total cost of ownership on the servers you purchase, you should ensure efficient utilization of the system, which means the system must achieve and sustain near-80 percent CPU utilization during the worst failure mode at peak load. Here are four ways you can efficiently utilize the processors available today:
Increase the workload, by deploying more active mailboxes per server.
Introduce a virtualization layer, and deploy the mailbox role as a guest machine, along with additional guest machines.
Deploy additional Exchange server roles onto the system.
Use a combination of the above methodologies to find the optimal configuration that utilizes the hardware as efficiently as possible.
Deploying Exchange 2010 with a multiple-role architecture provides several benefits:
The multiple-role architecture becomes a building block-based architecture. With the multiple-role architecture, all servers in the Exchange environment (excluding Unified Messaging and Edge Transport) are exactly the same—the same hardware, the same configuration, and so forth. This uniformity simplifies ordering the hardware, as well as performing maintenance and management of the servers.
- From a cost perspective, the overall goal is to ensure that the architecture is balanced from both a CPU perspective and from a disk perspective. Deploying server roles on separate machines can result in long-term cost disadvantages as you may purchase more CPU, disk, and memory resources than you will actually use. For example, consider a server that hosts only the Client Access server role. Many servers enable you to add a given number of disks in a very economical fashion—when you are deploying that number of disks and, more importantly utilizing them, the cost is essentially zero. But if you deploy a server role that uses far less than the given number of disks, you’re paying for a disk controller that is either under-utilized or not utilized at all.
In many cases, using a multiple-role architecture enables you to have fewer physical Exchange servers in the environment. Fewer physical servers mean lower costs for a variety of reasons:
Operational expenditures are almost always higher than the capital expenditures. It costs more to manage a server over its lifetime than it does to purchase it.
You purchase fewer Exchange server licenses. A multiple-role server only requires a license for one Exchange server and one operating system, while breaking out the roles requires multiple Exchange server licenses and possibly multiple operating system licenses. For more information, see About Licensing: Licensing for Virtual Environments.
Deploying fewer servers has a trickle-down effect across the rest of the infrastructure. For example, deploying fewer physical servers may reduce the total rack and floor space required for the Exchange infrastructure, which in turn reduces power and cooling costs.
A multi-role architecture ultimately distributes the load across a greater number of servers than deploying single-role servers because all Mailbox servers also become Hub Transport servers and Client Access servers. This architecture provides two benefits:
From a scalability perspective, you’re distributing the load across a greater number of physical machines. During a failure event, the increased load on the remaining servers only increases incrementally, which ensures the other functions the server is performing aren’t adversely affected.
From a resiliency perspective, the solution can survive a greater number of Hub Transport and Client Access role (or service) failures and still provide service.
For these reasons, the deployment strategy we recommend for Exchange 2010 is a multiple-role server configuration for most scenarios.
When Multiple-Role Configurations are Recommended
Multiple-role configurations are recommended for most scenarios for the following reasons:
Simple unit of scale Organizations that anticipate regular growth in the number of mailboxes should consider deploying multiple-role servers. Because each multiple-role server represents a building block, this model allows the easy addition of building blocks to support the need for increased capacity.
Large-scale deployments that want to leverage modern processors Based on scalability testing performed prior to the release-to-manufacturing (RTM) version of Exchange 2010, multiple-role servers can effectively utilize hex core (or more) processors in a single server. This capability allows large organizations to reduce the number of servers by combining the Mailbox, Hub Transport, and Client Access server roles instead of deploying these roles separately on servers with fewer processor cores. This approach leverages the building block model described earlier to provide a platform for large-scale deployments while reducing the overall number of servers required. Scalability of the multiple-role configuration on larger core count systems should be validated with lab testing prior to production deployment.
Server deployments with internal storage Many servers available today have two physical multi-core processors and 10 to 16 internal disks. Several improvements in Exchange 2010 reduce I/O requirements, making these servers a cost-effective solution. Depending on user profile and disk type, these servers generally support up to 4,000 mailboxes. We recommend adding the Client Access and Hub Transport server roles to these servers to utilize the additional CPU and make these servers self-contained building blocks.
Risk mitigation scenarios where the number of mailboxes hosted on a Mailbox server is limited Multiple-role servers are a solution for deployments where risk management policies limit the number of mailboxes that can be deployed on a Mailbox server. For example, say an organization with 10,000 mailboxes has a policy that a single server outage can't affect more than 25 percent of the mailboxes in the environment. This requirement limits the number of mailboxes per Mailbox server to 2,500. The additional capacity on that server could be utilized by adding the Client Access and Hub Transport server roles to the server.
Small organizations and branch office deployments Except as noted below when Windows Network Load Balancing is used, a multiple-role deployment is a recommended solution for deployments where the primary goals are to minimize the number of physical servers, operating system instances, and Exchange servers to manage. Running the Client Access, Hub Transport, and Mailbox server roles on the same physical server provides the necessary role redundancy with a minimum requirement of two or three physical servers.
Return to top
When Multiple-Role Configurations Aren't Recommended
Multiple-role configurations aren't recommended for the following scenarios:
Small organizations, or branch office deployments, that want to use Windows Network Load Balancing (NLB) Multiple-role servers may not work well for small deployments where two or three multiple-role servers are being deployed as members of a database availability group (DAG). For more information about DAGs, see Managing Database Availability Groups. The clustering component added to Mailbox servers that are members of a DAG prevents NLB from being installed on the server. For more information about load balancing recommendations, see Understanding Load Balancing in Exchange 2010. However, there's still a requirement to load balance inbound traffic to the Client Access servers. In this case, there are two main options:
Purchase a hardware load balancing appliance. Although there are some entry-level NLB appliances, this option can be costly, especially for smaller environments.
Virtualize the Exchange server roles. In some environments, a limited number of servers results in having to deploy domain controllers, file and print servers, and other applications on the same physical hardware as the Exchange 2010 servers. We recommend that you implement the physical servers as host servers and isolate applications inside a virtual environment. With this isolation, you can run NLB for Client Access servers running on virtual machines.
Virtualization The maximum number of active mailboxes that can be hosted by a virtual machine may be reduced based on the combination of message profile and running in a multi-role configuration. If you have light messaging users, co-locating server roles in a virtual machine may make sense. However, if you have heavy messaging users, you may be limited for resources in a virtual machine, and thus you may need to either reduce the number of mailboxes per Mailbox virtual machine or split out the roles into separate virtual machines. In these cases, it may be more efficient to deploy a single Exchange server role in each virtual machine, or to deploy one combined Client Access and Hub Transport virtual machine for every Mailbox server virtual machine.
Note
You can’t install an Exchange server role on the hypervisor host server. Only management software (for example, antivirus software, backup software, or virtual machine management software) can be deployed on host servers. No other server-based applications should be installed on the host server (for example, Exchange, Microsoft SQL Server, or Active Directory). The host servers should be dedicated to running guest virtual machines.
For more information, see Understanding Client Access and Hub Transport Combined Role Configurations in Capacity Planning.
Return to top
Hardware Recommendations for Multiple-Role Servers
As a general guideline, a multiple-role server should be sized to use half of the available processor cores for the Mailbox server role and the remaining half for the Client Access and Hub Transport server roles. Microsoft doesn't specify a maximum number of recommended processor cores for multiple-role servers. Instead, a maximum number of populated processor sockets are provided. This refers to the number of processor sockets on the motherboard where multi-core processors are connected. For more information, see Understanding Processor Configurations and Exchange Performance.
In addition to sizing the processor architecture, the memory must also be sized correctly for deploying a multiple-role configuration. For more information, see Understanding Memory Configurations and Exchange Performance.
Deploying a Multiple-Role Server in a DAG
When you're deploying single-role Mailbox servers in a DAG, consider capacity planning for single and multiple server failures in relationship to Mailbox server load. If you have four Mailbox servers in a DAG, size the Mailbox servers at 50 percent capacity so that they can accommodate double the number of active users, in the event of simultaneous failure of two Mailbox servers. Because the Hub Transport and Client Access servers are on different physical servers, the load on those servers isn't impacted much by the loss of one or two Mailbox servers.
When you're deploying multiple-role servers in a DAG, think about capacity planning for Client Access, Hub Transport, and Mailbox server load. If you have four multiple-role servers in a DAG, make sure there is sufficient capacity to accommodate a potential doubling of Hub Transport and Client Access server load. Because the multiple-role configuration aligns with the recommended processor core ratios for server roles, if you correctly size the maximum active databases for the Mailbox server role, Hub Transport and Client Access servers should meet the performance objectives for this scenario.
Return to top
Example of Sizing for an Exchange 2010 Multiple-Role Scenario
The following example illustrates the server sizing process for multiple-role servers. The example has the following design assumptions:
Total mailbox count 24,000
Mailbox profile 100 messages per day (for example, 20 sent and 80 received)
Database cache per mailbox 6 MB (based on a 100 message per day profile)
Availability requirements Mailbox resiliency within a single site; protection against simultaneous failure of three database copies and two servers
Database requirements 120 databases in the DAG, 200 mailboxes per database
Server platform 2 x 6 core 2.26 gigahertz (GHz) processor-based (X5650) server (12 cores)
The following process applies:
Calculate server count A four-node DAG is required to protect against the simultaneous failure of two servers. However, the customer has decided to deploy six servers to control the maximum number of active mailboxes during a double server failure event. Therefore, the design begins with six Mailbox servers within the DAG.
Calculate maximum active mailboxes per server based on the activation model Assuming the active databases are equally distributed across the nodes, each server ideally hosts 4,000 active mailboxes (24,000 ÷ 6). To calculate the active mailbox count after a double-node failure (based on this example), the mailbox count is divided by the remaining four nodes, which equals 6,000 active mailboxes per node (24,000 ÷ 4).
In this example, the MaximumActiveDatabases parameter on the Set-MailboxServer cmdlet is configured for 30 to ensure that no more than 40 percent of the databases become active on a single server.
Calculate active mailbox CPU requirements Multiply the maximum number of active mailboxes on a server by the megacycles per active mailbox (6,000 × 2 megacycles = 12,000 megacycles), based on the Estimated IOPS per mailbox based on message activity and mailbox database cache table in Understanding the Mailbox Database Cache. Multiply this value by 10 percent for each additional database copy.
In this example, there's one active copy and three passive copies for every database, so the 12,000 megacycles is increased by 30 percent (12,000 × 1.3 = 15,600 megacycles). For more information, see "Database Cache Metrics" in Understanding the Mailbox Database Cache.
Calculate passive mailbox CPU requirements Multiply the number of passive mailboxes (when a server is hosting the maximum number of active mailboxes) by the megacycles per passive mailbox (10,000 × 0.3 megacycles = 3,000 megacycles), based on the Estimated IOPS per mailbox based on message activity and mailbox database cache table in Understanding the Mailbox Database Cache. For more information, see "Database Cache Metrics" in Understanding the Mailbox Database Cache.
Add active and passive CPU requirements to get total CPU requirement In this example, 15,600 active mailbox megacycles + 3,000 passive mailbox megacycles = 18,600 megacycles total CPU requirement.
Apply Mailbox CPU requirement to hardware platform This example uses a 2 x 6 core 2.26-GHz processor-based (x5650) server. Based on the guidance in Mailbox Server Processor Capacity Planning, this equates to 60,083 megacycles. Divide the required megacycles by the available megacycles based on the server platform to estimate the CPU utilization at peak period after a double-node failure (18,600 ÷ 60,083 = 31 percent predicted CPU utilization).
We recommend that the Mailbox server role portion of multiple-role configurations be designed to not exceed 40 percent utilization during peak periods (for example, simultaneous failure of two nodes). This design allows sufficient space to accommodate CPU utilization of Client Access and Hub Transport server roles while maintaining total server CPU utilization at less than 80 percent during peak periods (for example, simultaneous failure of two nodes).
Calculate active mailbox memory requirements Multiply the number of active mailboxes by the required database cache per mailbox. In this example, with a double server failure, the remaining servers will host 6,000 active mailboxes (6,000 × 6 MB) ÷ 1,024 = 35.1 GB. The database cache requirements are based on the mailbox profile. For more information, see "Database Cache Metrics" in Understanding the Mailbox Database Cache.
Apply total memory requirements to hardware platform The total memory required is based on the database cache requirements and the server design (dedicated or multi-role). For more information, see the Default mailbox database cache sizes table in Understanding the Mailbox Database Cache. The total memory requirement for the multi-role server in this example is 52.2 GB ((4 GB + 35.1 GB) ÷ 0.75). Because 52.2 GB isn't a standard memory configuration, round up to 64 GB or the closest memory configuration that your server supports.
Return to top
© 2010 Microsoft Corporation. All rights reserved.