Understanding Node Roles in Microsoft HPC Pack
Applies To: Microsoft HPC Pack 2008 R2, Microsoft HPC Pack 2012, Microsoft HPC Pack 2012 R2
In Microsoft® HPC Pack, the node role determines the functionality of a node. A node can perform functions such as managing the cluster, running jobs, or routing SOA service requests. The role or roles that a node can perform are determined when you install HPC Pack (when you configure or deploy a node). Head nodes and broker nodes can perform more than one node role at a time. If the required services and features are installed on the node, you can enable or disable the Compute Node or Broker Node roles according to your cluster workload and size. For more information, see the Multiple roles and broker scaling section in this topic.
The node role determines a node’s inclusion in the built-in node groups: HeadNodes, ComputeNodes, WCFBrokerNodes, WorkstationNodes, UnmanagedServerNodes, and AzureNodes (changed from AzureWorkerNodes). You can use the node group property to filter and select nodes so that you can monitor and perform operations on nodes according to their roles, or submit jobs to nodes in specific groups.
The following table describes the node roles:
Node Role |
Functionality |
Normal node state |
|---|---|---|
Head node |
Provides management and job scheduling services to the cluster. Listens for session requests from Service-Oriented Architecture (SOA) client applications across the Enterprise network and provides the client with information for connecting to a WCF Broker Node. Must be connected to the Enterprise network. |
Offline |
WCF Broker node |
Receives requests from a SOA client, distributes them to the service hosts on the cluster (nodes), and then collects responses and sends them back to the client. If the session is a DurableSession, the broker stores the messages using Message Queuing (MSMQ). Monitors and manages SOA sessions. Must be connected to the Enterprise network. |
Online |
Compute node |
Accepts and runs cluster jobs. Hosts SOA services. Also includes:
|
Online |
Workstation node |
Accepts and runs cluster jobs. Hosts SOA services. Also includes:
|
Online or Offline (per Availability Policy) |
Windows Azure Worker Node |
Role available starting with HPC Pack 2008 R2 with Service Pack 1 (SP1). Accepts and runs cluster jobs. Hosts SOA services. Also includes:
|
Online or Offline (per Availability Policy) |
Windows Azure Virtual Machine Node |
Deprecated. Important The VM Role feature (beta) in Microsoft Azure is being retired on May 15, 2013. Also now deprecated are the settings in Microsoft HPC Pack 2008 R2 and Microsoft HPC Pack 2012 to deploy a custom VHD to VM role nodes from a Windows HPC cluster. After the retirement date, VM role deployments from an HPC cluster will fail or be inaccessible. To add Azure nodes to an HPC cluster, use the Azure worker role. |
|
Unmanaged Server Node |
Role available starting with HPC Pack 2008 R2 with Service Pack 3 (SP3). Other servers that can be used opportunistically to run cluster jobs during periods of low demand for their typical network workloads, such as running a file server or print server. |
Offline |
Multiple roles and broker scaling
The head node includes the features and services for the broker node and compute node role. If the head node is not configured in a failover cluster (for high-availability), it can perform one or both of these additional roles. By default, the head node has all three roles enabled, so if you bring it to the Online node state, it will start performing all three roles. Running all three roles on the head node allows you to set up a development ‘cluster’ with a single server. Two roles on the head node can be useful to maximize resource usage in smaller clusters if you do not need an entire server dedicated to management and job scheduling.
Broker nodes include the features and services for the compute node role, so you can quickly switch between roles. For larger clusters that support SOA workloads, this allows you to scale your compute nodes and broker nodes as your workload requires. For example, the number of SOA sessions that your cluster supports might vary according to weekly or monthly business cycles, or might slow down at night and on weekends. You can enable more broker nodes when demand for sessions is high, and then convert them to compute nodes when demand slows down.
Note
When the compute node role is enabled on a node, and the node is Online, the HPC Job Scheduler Service considers every core on the node as a potential job resource. The HPC Job Scheduler Service tries to keep the cluster as busy as possible, so it will attempt to allocate tasks to every core. If you enable compute node as an additional role, the head node and/or broker node functionality will have to share CPU time with running tasks. The feasibility of adding the compute node role depends on how busy the head node or broker node are, and how much work the tasks are doing.
Service Pack 2 includes the ability to specify the number of cores on a node that you want to make available to the HPC Job Scheduler Service. If you want to enable multiple roles on a node, you can use this feature to essentially limit the number of cores that will be used for the compute node role. For more information, see Over-subscribe or under-subscribe core or socket counts on cluster nodes.
How to change node roles
To enable or disable node roles, take the node to the Offline state, right-click the node, and then click Change Role. The following screenshot illustrated how to disable the Compute Node role on the head node. In the screenshot, notice also that the node list view is configured to display node properties related to node role. For more information, see the How to keep track of installed roles and enabled roles section later in this topic.
.jpeg "Disable ComputeNode role on Windows HPC Head Node")
You can also change node roles using HPC PowerShell. You must be a cluster administrator to run the node management cmdelts, and you must run the cmdlets in an elevated HPC PowerShell window (right-click HPC PowerShell and then click Run as administrator). The following sample cmdlets demonstrate how to change node roles on a head node named myHeadNode and on two broker nodes named BN-01 and BN-02.
Sample cmdlets to take the nodes Offline using Set-HpcNodeState:
To take the head node offline
Set-hpcnodestate –name myHeadNode –state offline
To take the broker nodes offline:
Set-hpcnodestate –name BN-01,BN-02 –state offline
Sample cmdlets to change roles using Set-HpcNode:
To disable all additional roles on the head node:
Set-hpcnode –name myHeadNode –role none
To enable the broker node role on the head node:
Set-hpcnode –name myHeadNode –role brokernode
To enable both additional roles on the head node:
Set-hpcnode –name myHeadNode –role computenode, brokernode
SP2 To enable the compute node role on the head node and specify that only 2 cores should be used for jobs:
Set-hpcnode –name myHeadNode –role computenode –subscribedcores 2
To enable the compute node role on two broker nodes:
Set-hpcnode –name BN-01,BN-02 –role computenode
To enable the broker node role on two broker nodes:
Set-hpcnode –name BN-01,BN-02 –role brokernode
SP2 To enable both roles on a broker node and specify that only 2 cores should be used for jobs:
Set-hpcnode –name BN-01 –role brokernode,computenode –subscribedcores 2
Sample cmdlets to bring the nodes Online using Set-HpcNodeState:
To bring the head node online
Set-hpcnodestate –name myHeadNode –state online
To bring the broker nodes online:
Set-hpcnodestate –name BN-01,BN-02 –state online
How to keep track of installed roles and enabled roles
To determine which roles are enabled on a particular node, you can look at the node Groups property in the Details Pane. Alternately, you can filter the node list by selecting an option in the Navigation Pane under By Group.
You can add columns to the Node Management list view in HPC Cluster Manager to view the installed roles and the enabled roles. The Installed Service Roles property lists the roles that are installed. The Node Roles property lists the roles that are enabled (the Node Role column does not list the head node role). To add columns to the node list, right-click a column header and then click Column Chooser.
Note
If your head node is configured in a failover cluster, you can also add the PrimaryHeadNode property to the node list view to help keep track of which head node is the active head node.
The screenshot in the previous section shows the node list filtered by the HeadNodes group, and the node role columns displayed in the node list.