Share via

Configuring WCF Broker Nodes

  • Article

 

Applies To: Microsoft HPC Pack 2008 R2, Microsoft HPC Pack 2012, Microsoft HPC Pack 2012 R2

Using the service-oriented architecture (SOA) programming model, solution developers and architects can develop high-performance computing (HPC), cluster-enabled, interactive applications that enable distributed computing scenarios. To use a SOA application with your cluster, you must configure at least one node to be a Windows Communication Foundation (WCF) broker node. The broker node must be able to communicate with the client application and with the compute nodes (for more information about how SOA jobs work on a cluster, see SOA job).

This topic includes:

  • WCF Broker Node basics

  • Broker node templates

  • Pre-configuring a broker node

  • Durable Sessions and MSMQ

  • Limiting the number of concurrent session on a broker node

  • Scaling broker nodes

  • Additional considerations

WCF Broker Node basics

You can use the head node as a broker node, or you can deploy one or more nodes as broker nodes. Broker nodes monitor and manage sessions, and coordinate service request and responses between the SOA client and the service hosts.

Important

WCF Broker Nodes can only be used to manage SOA sessions if they are in the Online node state.

Note

To use the head node as a broker node, you must bring the head node to the Online node state. By default, when you bring the head node Online, it will also start acting as a compute node. You can use the Change node role action to disable the compute node functionality. For information about changing a node role, see Scaling broker nodes.

To perform as a WCF Broker Node, a server must meet the following requirements:

Broker node templates

To add a new broker node to the cluster, you need to create a broker node template. When the head node detects new nodes on the network, the nodes appear in HPC Cluster Manager in the Node Management view. The cluster administrator approves the nodes and joins them to the cluster by assigning a node template.

You can use the node template wizard in HPC Cluster Manager to create broker node, compute node, or workstation node templates. When the template is assigned, the corresponding node role is enabled automatically.

If you want to deploy your broker node from bare metal, you can create a broker node template that includes an associated operating system image. If you want to add a server that has already been configured as a broker node, you can create a broker node template that does not include an associated operating system image. For more information, see Node Templates.

Pre-configuring a broker node

You can install Microsoft® HPC Pack directly on a server to create a preconfigured broker node. The server must have a supported 64-bit edition of the Windows Server operating system installed.

When you first install HPC Pack on a server, you have the option to set up a head node, a compute node, or a broker node. The selection that you make in the wizard determines the features that are installed on the node. When you select the option to join a cluster by creating a broker node, the installation wizard includes a step to specify the head node of the cluster.

When the installation completes, the new node appears in the Node Management list view as WCF Broker Node in the Unknown state. You can then approve the node and add it to the cluster by assigning a broker node template that does not include an operating system image.

Durable Sessions and MSMQ

When SOA clients create sessions on the cluster using the Durable Session APIs, the broker node stores messages using Message Queuing (MSMQ). Responses that are stored by the broker can be retrieved by the client at any time, even after intentional or unintentional disconnect. By default, the MSMQ storage limit is 8 GB. Depending on the message size and the number of durable sessions running on your cluster, you can modify the record retention period or increase this limit. The amount of disk space that messages for durable sessions use can be estimated as follows:

Required disk space = (number of requests x request size) + (number of responses x request size)

In HPC Cluster Manager, you can monitor the total message queue size (requests and responses) and free disk space by adding columns to the node list view or creating a heat map page for the Durable Queues Total Bytes and Free Disk Space metrics.

You can adjust message queuing limits on the broker nodes in Server Manager under Features. Right-click Message Queuing and click Properties to modify the storage limits, as illustrated in the following screenshot:

In Server Manager Features, edit Message Queuing

During normal operations, messages are deleted from the MSMQ in the following ways:

  • SOA clients can cause the MSMQ to delete the messages associated with their session when they close the session.

  • Messages associated with finished sessions are deleted automatically from the MSMQ after the number of days specified in the Job History settings in the job scheduler configuration options. By default, records for completed sessions are kept for five days. See also Understanding Job History.

Limiting the number of concurrent session on a broker node

You can place a limit on the number of concurrent sessions on a broker node. When the defined limit is reached, the following occurs when a SOA client tries to connect:

  • New session: the SOA client tries the next broker node. If all broker nodes are full, the session creation fails.

  • Attach to session: the attach operation fails immediately because the broker node that is managing the specified session is already at capacity.

To set the limit, you can edit the value of the MaxConcurrentSession attribute in the HpcbBoker.exe.config file on each broker node. This file is located in the %CCP_HOME%Bin folder. The default value for MaxConcurrentSession is 0, which means that the setting is disabled, as illustrated in the following XML code snip:

<userSettings>
  <Microsoft.Hpc.Scheduler.Session.Internal.BrokerLauncher.BrokerLauncherSettings>
    <setting name="MaxConcurrentSession" serializeAs="String">
      <value>0</value>
    </setting>
  </Microsoft.Hpc.Scheduler.Session.Internal.BrokerLauncher.BrokerLauncherSettings>
</userSettings>

Scaling broker nodes

Changing node roles allows you to easily scale your compute nodes and WCF broker nodes depending on your current needs. When a node includes the features for multiple roles, you can enable and disable the node roles in HPC Cluster Manager. When a node is deployed as a broker node, the node includes the features for the broker node role and the compute node role. For more information, see Understanding Node Roles in Microsoft HPC Pack.

The following procedure describes how to enable the WCF broker node role:

To enable the WCF broker node role

  1. In Node Management, in the Navigation Pane, click Nodes.

  2. In the views pane, right-click a node, then click Take Offline.

  3. In the Take Offline Node dialog box, click Yes. Look at the node status in the views pane, and wait until the state of the node has changed to Offline before going to the next step.

  4. In the views pane, right-click the node, then click Change Role.

  5. In the Change Node Role dialog box, select WCF broker node, and then click OK.

  6. In the views pane, right-click the node, and then click Bring Online.

  7. In the Navigation Pane, under Nodes, under By Group, click WCFBrokerNodes. Confirm that the node you configured appears in the list.

Additional considerations

  • You can optimize the job scheduler resource allocation policy for SOA workloads. For more information, see Understanding Policy Configuration.

  • Developers create a service DLL, a service configuration file, and a client application. Administrators copy the service DLLs to compute nodes on the cluster and register the service by placing the service configuration file on the head node. For information about deploying and configuring SOA services, see Managing SOA Services in Microsoft HPC Pack.

  • The service configuration file can be used to customize how the broker interacts with a particular service. For example, broker behavior such as monitoring, message throttling, and load balancing can be defined at the individual service level. For more information, see SOA Applications, Infrastructure and Management.

  • The client application provides an interface for the end-user, and on the back-end the client application submits a SOA job to the cluster, initiates a session with the broker node, and sends service requests and receives responses. End-users run the application on their computers and cluster nodes perform calculations. The session ID corresponds to the job ID.

  • For more information about WCF broker nodes and SOA applications, see the SOA Applications, Infrastructure and Management white paper.

  • For information about using HPC Cluster Manager, see Overview of HPC Cluster Manager.

Additional references