What's New in Microsoft HPC Pack 2012 R2 Update 1

Article
11/13/2015

Applies To: Microsoft HPC Pack 2012, Microsoft HPC Pack 2012 R2

This document lists the new features and changes that are available in Microsoft® HPC Pack 2012 R2 Update 1. For late-breaking issues and getting started guidance, see Release Notes for Microsoft HPC Pack 2012 R2 Update 1.

In this topic:

Azure IaaS deployment and management
Azure PaaS (“burst”) deployment enhancements
Installation
Node management
Job scheduling
Runtimes and development

Azure IaaS deployment and management

HPC Pack image now available in the Azure Marketplace on both the Azure global and Azure China services For this release, we added a new HPC Pack virtual machine image to the Azure Marketplace. This Azure VM image is based on Windows Server 2012 R2 Datacenter Edition with HPC Pack 2012 R2 Update 1 pre-installed. Microsoft SQL Server 2014 Express is also pre-installed. You can use this image to create the head node of a Windows HPC cluster in Azure. We recommend using a VM size of at least A4. Before creating a VM, you must configure a valid virtual network for it if the head node is not planned as a single-machine scenario. Additionally, while or after creating the VM, you will need to join the virtual machine to an Active Directory domain. You can optionally promote the head node as the domain controller. For more information, see Create a Head Node from the Microsoft HPC Pack Azure VM Image.
HPC Pack IaaS deployment script to deploy an HPC cluster on Azure VMs The HPC Pack IaaS deployment script provides an easy and fast way to deploy HPC clusters in Azure infrastructure services (IaaS) on either the Azure Global or the Azure China service operated by 21Vianet. It utilizes the HPC Pack VM image in the Azure Marketplace for fast deployment and provides a comprehensive set of configuration parameters to make the deployment easy and flexible. The HPC Pack IaaS cluster can be set up from a client computer by using only the Azure subscription ID and the management certificate or publish setttings file; the script can then create all the cluster infrastructure, including the virtual network, affinity groups, storage accounts, cloud services, domain controller, remote or local SQL databases, head node, broker nodes, compute nodes, and Azure PaaS (“burst”) nodes. The script can also use pre-existing Azure infrastructure and then create the HPC cluster head node, broker nodes, compute nodes and Azure PaaS nodes. The HPC Pack web portal and REST API can also be enabled by the script. For details, see Create an HPC cluster with the HPC Pack IaaS Deployment Script.

Note

Before running this script, you must configure an Azure publish settings file or an Azure management certificate on the client.
HPC Pack IaaS management scripts to add, remove, start, or stop Azure VM nodes in an HPC Pack IaaS cluster Besides the easy and fast deployment script, we provide a set of management scripts on the IaaS head node to help customers easily manage the Azure VM compute nodes in an HPC Pack IaaS cluster. With Add-HpcIaaSNode.ps1 and Remove-HpcIaaSNode.ps1, users can easily expand or shrink the number of IaaS VMs with different customized compute node images and different VM sizes in multiple cloud services and storage accounts. With Start-HpCIaaSNode.ps1 and Stop-HpcIaaSNode.ps1, you can grow and shrink the number of Azure VM compute nodes according to the workload. When the number of jobs or tasks increases, start the VMs to bring the compute nodes online to grow the compute resources. When the workload finishes, bring the compute nodes offline and stop the VMs into the Stopped (Deallocated) state to save costs. For details, see Manage the Compute Node VMs in an HPC Pack IaaS Cluster.

Note

Before running these scripts, you must configure an Azure publish settings file or an Azure management certificate on the client.

Azure PaaS (“burst”) deployment enhancements

Automatically detect and support new node sizes In HPC Pack 2012 R2 and prior releases, the Azure node (worker role instance) sizes supported in burst deployments are fixed. HPC Pack 2012 R2 Update 1 allows newly added Azure worker role sizes that are supported by HPC Pack to be automatically refreshed and appear in the dropdown list in the Add Node Wizard when adding Azure burst nodes. The list of supported role sizes is maintained by Microsoft and is updated periodically as the platform supports new sizes. Depending on your Azure subscription, you may not be able to use all Azure role sizes.
Azure regional virtual networks supported Azure regional virtual networks (“wide” VNets) were introduced in May 2014 and newly created VNets now span an entire Azure region instead of a hardware cluster in an Azure data center. Because of this, a regional VNet is bound to a location instead of an affinity group. In HPC Pack 2012 R2 and prior releases, when creating Azure nodes template, if the VNet is required, HPC Pack only detects and validates the VNets associated with the affinity group of the specified cloud service. Starting in HPC Pack 2012 R2 Update 1, HPC Pack fully supports regional VNets for Azure burst deployments, and continues to support the previous “narrow” VNet that is associated with an affinity group for backward compatibility.
Start selected Windows Azure nodes HPC Pack 2012 R2 Update 1 allows you to start selected nodes from HPC Cluster Manager or the new Start-HpcAzureNode HPC PowerShell cmdlet. With this feature you can now add a set of Azure nodes once and then scale up the number of provisioned nodes to match the workloads in the job queue multiple times. Working with the stop selected Azure nodes feature introduced in HPC Pack 2012 R2, you can scale up and down the number of provisioned Azure nodes without adding and removing the nodes each time. In previous versions of HPC Pack, you can only start an entire set of nodes deployed with a particular node template.
Azure burst deployments supported in Azure China HPC Pack 2012 R2 Update 1 now supports Azure burst deployments in the Azure China service and provides a simple PowerShell script to run on the head node(s) to switch between the Azure Global and Azure China services. To switch between the Azure Global and Azure China service, on each head node, open PowerShell as an administrator, go to $env:CCP_HOME\bin, and run .\Update-HpcAzureEndpoint.ps1 -AzureType Global or .\Update-HpcAzureEndpoint.ps1 -AzureType China. You don’t need to restart any HPC services after running the script. To view the current Azure service used for the deployment, run .\Update-HpcAzureEndpoint.ps1 –View. For more information about the Azure China service operated by 21ViaNet, please refer to https://www.windowsazure.cn/zh-cn/support/faq/?l=zh-cn.

Note

You should stop any Azure burst deployments before changing the service type, or you may not be able to use the previous deployments.

Installation

Support for Microsoft SQL Server 2014 The HPC Pack 2012 R2 Update 1 installation package and the HPC Pack Azure VM image contain SQL Server 2014 Express to host local HPC cluster databases. If SQL Server 2012 or 2008 R2 Express is not detected on the head node, the installation package installs SQL Server 2014 Express. (The upgrade installation does not upgrade an existing SQL Server 2012 Express or 2008 R2 Express installation to SQL Server 2014 Express.) For remote databases, SQL Server 2014 Standard and Enterprise are now also supported. The remote databases can be configured with a SQL Server AlwaysOn failover cluster instance or availability group for high availability. For more information about SQL Server HA solutions, see High Availability Solutions (SQL Server).

Node management

Automatically grow or shrink the number of Azure nodes and Azure VMs according to workload.

HPC Pack 2012 R2 Update 1 provides the AzureAutoGrowShrink.ps1 PowerShell script to monitor the jobs or tasks in the queue to automatically grow or shrink the number of Azure PaaS (burst) nodes or the IaaS compute node VMs by starting or stopping the nodes. You can specify a set of parameters, e.g., the number of queued jobs or the number of active queued tasks required to grow per one node, the threshold of the number of queued jobs or the number of active queued tasks required to start grow, the grow/shrink check interval, the number of sequential idle confirmations for a node to shrink, and the node templates or job templates for the grow/shrink scope. This script can be found under %CCP_Home%bin folder on head nodes and clients. When running with compute node VMs in Azure, the script should run on the IaaS head node because it depends on Start-HPCIaaSNode.ps1 and Stop-HPCIaaSNode.ps1 to start and stop the IaaS VMs. For detailed usage, run Get-Help .\AzureAutoGrowShrink.ps1 –detailed in an elevated PowerShell window under $env:CCP_HOME\bin. For more information, see Grow and Shrink Azure Compute Resources in an HPC Pack Cluster.
Configure management operation log archive and retention intervals To avoid filling the HPC Management database with management operation logs, the previous fixed archive interval (7 days) and retention interval (180 days) are now configurable via the HPC PowerShell cmdlet Set-HpcClusterProperty with the –OperationArchive and –OperationRetention parameters. The current values can be viewed by Get-HpcClusterProperty.
Script support to move compute nodes, broker nodes, workstation nodes, and unmanaged server nodes to a different cluster or head node HPC Pack 2012 R2 Update 1 provides the Move-HpcNode.ps1 PowerShell script to easily move cluster nodes to a different head node without reinstalling or reimaging the nodes. The scripts can run either with or without the previous head node, in case the previous head node is unavailable. For more information, see Move Cluster Nodes to a New Head Node.

Note

Before moving WCF broker nodes, make sure there are no active SOA sessions on the nodes. After moving the broker nodes, the requests and responses for previous SOA sessions could be lost.

Job scheduling

Finish task operation added to finish active tasks directly to the Finished state

Similar to the existing job finish operation, a task finish operation is newly added to all users to finish a task. The task finish operation stops the task process immediately and marks the task as finished. HPC Pack 2012 R2 Update 1 supports this operation via the task finish command and the Stop-HpcTask HPC PowerShell cmdlet, and the HPC Job Scheduler API in the SDK.

Command line example:
```
task finish 1.1 /message: "Finish the task"
```
HPC PowerShell example:
```
Stop-Hpctask -JobID 5 -TaskId 1 -SubTaskId 5 -State Finished
```
API example:
```
Scheduler s = new Scheduler();
s.Connect("HeadNode");
var job = s.CreateJob();
job.Name = "EurekaTaskJob";
var task = job.CreateTask();
task.CommandLine = "ping localhost -n 60 && exit 1";
job.AddTask(task);
s.SubmitJob(job, "username", "password");
job.FinishTask(task.TaskId, "Finish the task.")
```

Finish or cancel a running job gracefully to let running tasks complete HPC Pack 2012 R2 Update 1 provides a new way to finish or cancel a running job gracefully, which lets the running tasks run until they are complete. You can do this via the job finish and job cancel commands, the Stop-HpcJob HPC PowerShell cmdlet, and the HPC Job Scheduler API in the SDK.

Command line examples:

job finish 2 /graceful /message: "Finish the job gracefully"
job cancel 5 /graceful /message: "Cancel the job gracefully"

HPC PowerShell examples:

Stop-HpcJob -Id 2 –State finished -Graceful –Message "Finish the job gracefully"
Stop-HpcJob -Id 2 -Graceful –Message "Cancel the job gracefully"

API example:

Scheduler s = new Scheduler();
s.Connect("HeadNode");
var job = s.CreateJob();
job.Name = "EurekaTaskJob";
var task = job.CreateTask();
task.CommandLine = "ping localhost -n 60 && exit 1";
job.AddTask(task);
s.SubmitJob(job, "username", "password");

//Finish method 1
job.Finish(isForced: false, isGraceful: true);
//Finish method 2
s.FinishJob(jobId: job.Id, message: "Finish the job gracefully", isForced: false, isGraceful: true);

//Cancel method 1
job.Cancel(isForced: false, isGraceful: true);
//Cancel method 2
s.CancelJob(jobId: job.Id, message: "Cancel the job gracefully", isForced: false, isGraceful: true);

Job-level automatic task requeue after application execution failure HPC Pack 2012 R2 Update 1 introduces an integer job property TaskExecutionFailureRetryLimit with default value 0. When set with a positive integer N, all the tasks in a job except node preparation and release tasks automatically requeue for a maximum of N times when an application execution failure happens (e.g., return code is not zero or not within the defined scope). This property is available in related job commands, HPC PowerShell cmdlets such as New-HpcJob, and the HPC Job Scheduler API. Previous versions of HPC Pack provide only cluster-wide automatic job or task requeue settings for a cluster scheduling or dispatching problem.

API example:
```
Scheduler s = new Scheduler();
s.Connect("HeadNode");
var job = s.CreateJob();
job.Name = "TestFailRetry";
job.TaskExecutionFailureRetryLimit = 3;
var task = job.CreateTask();
task.CommandLine = "ping localhost -n 5 && exit 1";
job.AddTask(task);
s.SubmitJob(job, "username", "password");
```
Scheduling jobs and tasks to unmanaged resources (Preview) HPC Pack 2012 R2 Update 1 introduces a Preview feature that developers can leverage to write their own resource communicator for the Windows HPC scheduler to support new types of compute resources that are not supported by default in HPC Pack. Examples include Linux nodes and Azure Batch service resources. We will release code samples in the near future.

Runtimes and development

Shortened SOA job rebalancing interval in Balanced scheduling mode The lower limit of allocationAdjustInterval in the SOA service configuration file is changed from 15000 ms to 3000 ms. You can use a lower setting for allocationAdjustInterval along with statusUpdateInterval (e.g., 5000 ms and 3000 ms) to shorten the SOA job rebalancing interval in Balanced scheduling mode. The statusUpdateInterval limits are unchanged. The change to allocationAdjustInterval does not affect any of your existing service configuration files unless you modify them explicitly.
MS-MPI runtime The v5 release of MS-MPI is included and installed in this release of HPC Pack. MS-MPI v5 includes the following features, improvements, and fixes:
- Improved the affinity algorithms for mpiexec.exe (/affinity and /affinity_layout options)
- Added support for MPI_Win_allocate_shared and MPI_Win_shared_query
- Added support for MPI_Comm_split_type
- Added support for MPI_Get_library_version
- Added support for configuring the queue depths used for Network Direct connections using the MSMPI_ND_SENDQ_DEPTH and MSMPI_ND_RECVQ_DEPTH environment variables
- Removed the dependency on the Visual Studio runtime
- Fixed incompatibility with Visual Studio 2012 and 2013 when building mixed Fortran and C++ applications
- Added SAL annotations to the APIs to better support static code analysis of MPI applications
- Refactored debugging support to implement MQS debugging as defined by the MPI Forum
Note

MS-MPI v5 can also be downloaded as a separate installation package, including the software development kit (SDK), from the Microsoft Download Center.