Set up Memory-Aware Scheduling
You can optionally configure the job scheduler to allocate a job to nodes taking into account the memory needs of the job and the available memory on the nodes. The goal is to give memory-intensive applications sufficient memory to run while at the same time enable less memory-intensive applications to run efficiently on other cluster resources.
In HPC Job Manager, the option to set up memory-aware scheduling is available in the New Job, Modify Job, and Copy Job dialog boxes. You can use the following procedure to create a new job using HPC Job Manager and then specify the Job properties specific to memory-aware scheduling.
Note
Memory-aware scheduling was introduced in HPC Pack 2012 and is not available in previous versions.
To enable memory-aware scheduling
In HPC Job Manager, in the Actions pane, click New Job.
In the left pane of the New Job dialog box, click Job Details, and then provide the following details for the job:
In the Job name text box, type a name for the job.
In the Job template drop-down list, select a job template. After you select a job template for a new job, the available values for job properties in the New Job dialog box change accordingly.
In the left pane of the New Job dialog box, click Edit Tasks.
Add tasks to your job:
To create a new task, point to the Add button, click the down arrow, then select a task type. In the task dialog box, provide information for your task, and then click Save to add the task to your job and return to the New Job dialog box.
To add a saved task to your job, point to the Add button, click the down arrow, and then click From Task File. Navigate to the location of the task XML file and click Open to add the task to your job and return to the New Job dialog box.
In the New Job dialog box, you can see a list of the tasks that you added.
Optionally, you can set the order in which the tasks are run. Click the Dependency button to open the Task Dependency dialog box. For more information, see Define Task Dependencies - Job Manager.
Optionally, in the left pane of the New Job dialog box, click Resource Selection to specify particular nodes, node groups, or hardware preferences.
Optionally, in the left pane of the New Job dialog box, click Licenses to specify any software licenses that your job requires.
Optionally, in the left pane of the New Job dialog box, click Environment Variables to specify environment variables to set in the run-time environment of all tasks in the job.
In the left pane of the New Job dialog box, click Advanced. In the Estimated memory per process section, to estimate the maximum amount of memory (in MB) that a process in the job will use, specify an integer value that is in the range specified for this property in the template for the job. A value of 0, if allowed, indicates that memory-aware scheduling is disabled.
Click Submit.
Additional considerations
The minimum, maximum, and default values of estimated memory per process are defined in the job template that is used for the job. Different values can be configured in different job templates. In the Default job template, the minimum, maximum, and default values are initially set to 0, which disables memory-aware scheduling.
When using memory-aware scheduling, the job scheduler compares the estimated memory per process to the total physical memory that is available on each node. If they are available, cluster nodes that have total physical memory that is at least the value of the estimated memory per process will be considered to run the job.
Depending on the job templates in use on the cluster and the cluster workloads, an administrator can control the extent to which jobs are restricted to running on nodes with memory that is at least the estimated memory per process. For example, in a typical job template, if the minimum and maximum values are non-zero amounts, and the default value is approximately the minimum average per core memory in the cluster, then by default a job will reserve memory proportional to the number of cores it uses. On the other hand, if any job template on the cluster allows the estimated memory process to be set to 0, a job based on that template could impede the memory-aware scheduling configured in other jobs.
The estimated memory per process of a job cannot be modified while the job is running.