Steps: Submit a Job with a Node Preparation and a Node Release Task
Updated: March 2011
Applies To: Windows HPC Server 2008 R2
This topic walks through an example of how to submit a job to a Windows HPC Server 2008 R2 cluster. The job copies an application to each node that is allocated to the job, runs the application against a set of 1,000 indexed files, and then cleans up the nodes when the job completes. The sample job includes a Node Preparation task, a Parametric Sweep task, and a Node Release task. For more information, see Understanding Task Types.
The procedures in this topic describe three methods to create and submit the sample job. To understand the commands in the procedures, review the information in Before you start.
In this topic:
Before you start
Submit the sample job by using a command prompt window
Submit the sample job by using HPC PowerShell
Submit the sample job by using HPC Job Manager
Submitting the sample job by using the scheduler APIs (New!)
Before you start
To understand the tasks and commands that are specified in the sample job, review the information in this section.
How the sample job works
The sample job runs as follows:
The Node Preparation task runs on each node that is allocated to the job. The task copies the application from a shared folder on the network to a local folder on each node that is allocated to the job.
Each subtask in the Parametric Sweep task runs an instance of the application against one of 1,000 indexed files that are stored in a shared folder on the network.
The Node Release task runs on each node as it is released from the job. The task deletes the application and any temporary files that the application produced from each allocated node.
The procedures uses placeholder names to specify the name of the application and the shared folders and files for the job. The following list describes the files and applications for the sample job:
The application is named myApp.exe.
myApp.exe is stored on a shared folder on the network that is named \\networkShare\bin.
myApp.exe accepts a single file as input.
The files that we want to pass to myApp.exe are a series of 1,000 indexed files that are named in0001.txt through in1000.txt.
The input files are stored on a shared folder on the network that is named \\datashare\userName.
myApp.exe creates temporary files in the working directory that are in the form myApp.temp.xxxxxxx, where xxxxxxx can be any string.
What you need to know about the asterisk (*)
In Aufgaben mit Unteraufgaben können Sie das Sternchen (*) als Platzhalter für den parametrischen Löschindex (in Parametrisches Löschen-Aufgaben) oder für die Unteraufgaben-ID (in den Aufgaben Dienst, Knotenvorbereitung und Knotenfreigabe) verwenden. For example, in a parametric sweep task with 1000 sub-tasks, the job scheduler interprets in*.txt as in1.txt in the first sub-task, and as in1000.txt in the last sub-task.
You can include more than one asterisk to indicate the minimum number of positions to use when expressing the sweep index or sub-task number. In the sample job, the indexed input files are named in0001.txt through in1000.txt, so the sample procedures specify the sweep index from 1-1000, and the input files in the form in****.txt.
In tasks that include sub-tasks, the job scheduler interprets the asterisk (*) as a placeholder before passing the command to the compute nodes. To run a command that uses an asterisk as a wildcard, include the caret (^) as an escape character. In the sample job, the Node Release task includes a command to delete all the files that start with “myApp.temp.” from the node. The command must include a caret as follows:
del myApp.tmp.^*
Without the caret, the command for the first instance of the node release task attempts to delete a file named myApp.tmp.1 from the compute node. With the caret, the command attempts to delete all files that start with “myApp.temp.” from the node (myApp.temp.*).
Commands that are submitted from a command prompt window are additionally interpreted before they are passed to the job scheduler. For more information, see the Submit the sample job by using a command prompt window section in this topic.
^ Top of page
Submit the sample job by using a command prompt window
You can use the job command to create and submit jobs to a Windows HPC cluster from a command prompt window. In the following example, the ID of the new job is 42.
job new
job add 42 /type:nodeprep copy /Y \\networkShare\bin\myApp.exe c:\Users\userName\bin
job add 42 /parametric:1-1000 /workdir:c:\Users\userName\bin myApp.exe \\datashare\userName\in****.txt
job add 42 /type:noderelease /workdir:c:\Users\userName\bin del /Q myApp.exe ^& del /Q myApp.tmp.^^^*
Commands that are submitted from a command prompt window are interpreted before they are passed to the job scheduler. The job scheduler also interprets commands before passing them to the compute nodes. You can use one or more carets (^) as escape characters to control when special characters are interpreted.
In the sample node release task, note the following uses of the caret (^):
The
^&tells the command prompt not to interpret the&character. Without the caret, the command attempts to delete a file named myApp.tmp.^* from the local computer, and also adds a task to job 42 that only deletesmyApp.exe. With the caret, the command adds a task to job 42 that deletes the application and the temporary files.The
^^^*tells the command prompt not to interpret the second caret and not to interpret the asterisk (*). The command prompt passes^*to the job scheduler. The caret in front of the asterisk is necessary so that the job scheduler, in turn, does not interpret the asterisk as a placeholder for the sub-task ID.
To monitor the progress of your job, see Verification: Check Job Completion and Output.
^ Top of page
Submit the sample job by using HPC PowerShell
You can use the New-HpcJob, Add-HpcTask, and Submit-HpcJob cmdlets to create and submit jobs to a Windows HPC cluster by using HPC PowerShell. To open HPC PowerShell, click Start, point to All Programs, click Microsoft HPC Pack 2008 R2, and then click HPC PowerShell.
In the following example, the variable $j stores the new job ID so that you do not have to specify the job ID for each cmdlet.
$j=New-HpcJob
Add-HpcTask –job $j –Type NodePrep –Command "copy /Y \\networkShare\bin\myApp.exe c:\Users\userName\bin"
Add-HpcTask –job $j –Type ParametricSweep –start 1 –end 1000 –WorkDir c:\Users\userName\bin –command "myApp.exe \\datashare\userName\in****.txt"
Add-HpcTask –job $j –Type NodeRelease –WorkDir c:\Users\userName\bin –Command "del /Q myApp.exe & del /Q myApp.tmp.^*"
Submit-HpcJob –id $j
.gif) Note |
|---|
| In the Node Release task, use the caret (^) as an escape character in front of the asterisk (*) so that the job scheduler does not interpret the asterisk as a placeholder for the sub-task ID. |
To monitor the progress of your job, see Verification: Check Job Completion and Output.
^ Top of page
Submit the sample job by using HPC Job Manager
To open HPC Job Manager, click Start, point to All Programs, click Microsoft HPC Pack 2008 R2, and then click HPC Job Manager.
To create and submit the sample job
In HPC Job Manager, in the Actions pane, click New Job.
In the left pane of the New Job dialog box, click Edit Tasks.
Add the Node Preparation task:
Point to the Add button, click the down arrow, and then click Node Preparation Task.
In the task dialog box, in Command line, type the following command:
copy /Y \\networkShare\bin\myApp.exe c:\Users\userName\binClick OK to add the task to your job. The new task now appears in the list of tasks for your job.
Add the Parametric Sweep task:
Point to the Add button, click the down arrow, and then click Parametric Sweep Task.
In the task dialog box, specify the number of steps for the sweep task by setting the Start value to 1 and set the End value to 1000.
In Command line, type the following command (relative to the working directory):
myApp.exe \\datashare\userName\in****.txtIn Working directory, type the following:
c:\Users\userName\binVerify your index series under Step 4. Expand the Command Line column to see the full commands for each subtask. For this task, the preview pane displays the following:
Task.exe myApp.exe \\datashare\userName\in0001.txt Task.exe myApp.exe \\datashare\userName\in0002.txt…Task.exe myApp.exe \\datashare\userName\in1000.txt
Click OK to add the task to your job.
Add the Node Release task:
Point to the Add button, click the down arrow, then click Node Release Task.
In the task dialog box, in Command line, type the following command:
del /Q myApp.exe & del /Q myApp.tmp.^*NoteUse the caret (^) as an escape character in front of the asterisk (*) so that the job scheduler does not interpret the asterisk as a placeholder for the sub-task ID. In Working directory, type the following:
c:\Users\userName\binClick OK to add the task to your job.
Click Submit.
To monitor the progress of your job, see Verification: Check Job Completion and Output.
^ Top of page
Submitting the sample job by using the scheduler APIs
The following C# code snip illustrates how to create and submit the sample job by using the Microsoft.Hpc.Scheduler APIs.
try {
using (IScheduler scheduler = new Scheduler()) {
try {
scheduler.Connect(hostname);
} catch (Exception e) {
Console.Error.WriteLine("Unable to connect to " + hostname);
Console.Error.WriteLine(e.Message);
return -1;
}
ISchedulerJob myjob = scheduler.CreateJob();
myjob.AutoCalculateMax = true;
myjob.AutoCalculateMin = true;
// Node Prep task
ISchedulerTask task = myjob.CreateTask();
task.Type = TaskType.NodePrep;
task.CommandLine = "copy /Y \\networkShare\bin\myApp.exe c:\Users\userName\bin";
myjob.AddTask(task);
// Parametric task
ISchedulerTask task2 = myjob.CreateTask();
task2.Type = TaskType.ParametricSweep;
task2.WorkDirectory = "c:\Users\userName\bin";
task2.EndValue=1000;
task2.CommandLine = "myApp.exe \\datashare\userName\in****.txt";
myjob.AddTask(task2);
// Node Release task
ISchedulerTask task3 = myjob.CreateTask();
task3.Type = TaskType.NodeRelease;
task3.WorkDirectory = "c:\Users\userName\bin";
task3.CommandLine = "del /Q myApp.exe & del /Q myApp.tmp.^*";
myjob.AddTask(task3);
scheduler.SubmitJob(myjob, null, null);
}
} catch(Exception e1){
Console.WriteLine(e1.Message);
Console.WriteLine(e1.StackTrace);
}
^ Top of page
Additional considerations
Um diese Schritte ausführen zu können, müssen Sie dem Cluster als Benutzer hinzugefügt worden sein, oder Sie müssen Mitglied einer Domänengruppe sein, die dem Cluster als Benutzer hinzugefügt wurde. Weitere Informationen erhalten Sie bei Ihrem Clusteradministrator.
To submit jobs to a cluster, you must have the HPC Pack 2008 R2 client utilities installed on your computer. Um eine Verbindung zu einem Cluster herzustellen, muss der Computer Mitglied der gleichen Domäne oder einer Domäne sein, die über eine Vertrauensstellung zur Clusterdomäne verfügt.
The procedures use the default job template to create the job. Clusteradministratoren können Berechtigungen dazu angeben, welche Benutzer eine bestimmte Auftragsvorlage verwenden können. Informationen zu den Auftragsvorlagen, auf die Sie zugreifen können und die Sie verwenden sollten, erhalten Sie von Ihrem Clusteradministrator.