Share via

Security Considerations for File and Folder Permissions in Windows HPC Server 2008 R2

  • Article

Updated: February 2011

Applies To: Windows HPC Server 2008 R2

There are a number of files and folders on computers in an HPC cluster that are important for system security. For example, job submission and job activation filters are important because they run with the permissions and privileges of the System account, which means that someone who can change and run a filter can gain control of the system. This topic lists files and folders that are important, describes why they are important, and tells how their location is determined.

To help prevent an unauthorized user from changing important files, track and protect the computers and folders where these files are stored, and as appropriate, limit permissions on the files themselves. For example, if HPC cluster administrators export node templates to XML files, track and protect the computers and folders where these XML files are stored, and as appropriate, limit permissions on the XML files.

In this topic

Files that contain saved or exported information in XML format

Files with a location that is determined during an installation or configuration process

Table of shared folders that are located on the head node

Files in temporary folders on workstation nodes

Specialized files for HPC session configuration, power control, and unattended installation

Log files that contain detailed information such as trace information from SOA sessions

Files that contain saved or exported information in XML format

The following XML file types are important in an HPC cluster because they store information about nodes, jobs, tasks, or diagnostics. These files are often used to “export” information, that is, to save information that can be imported later for use in a different environment (such as for testing or production).

Important
With , you (an HPC administrator) can create all of the following types of XML files, and users can create job or task XML files. When you create such files, you can choose the locations where they are stored, and you can move or copy them to other locations, increasing their usefulness but also increasing their vulnerability. Therefore, for the reasons described in the following list, it is important to review and track the way you store these XML files.

  • Node templates (exported as XML files)

  • Node XML files

  • Job and task XML files

  • Job templates (exported as XML files)

  • Custom diagnostic tests (saved as XML files)

Node templates (exported as XML files)

This section describes node templates that are exported as XML files.

Note
Node templates that are exported as XML files are different from “node XML files” (described in the next section).

  • Security considerations: Node templates work with associated files such as images, driver files, and custom disk-partitioning files (Diskpart.txt files), to control the installation and Mayntenance of compute nodes. When you create node templates by using HPC Cluster Manager, they are stored in one of the SQL Server databases that support the cluster, but node templates can also be exported into an XML file format. Protecting node-template XML files, and associated files such as images, drivers, and custom Diskpart.txt files, helps ensure that the nodes will deploy as expected. It also helps protect against installation of malicious software (malware) or other unintended software on compute nodes.

    Important
    For information about the location of associated files such as images, driver files, and custom Diskpart.txt files (which are all important for the security of installed nodes), see Table of shared folders that are located on the head node, later in this topic.

  • How node templates are exported or imported as XML: In HPC Cluster Manager, two of the actions that are available when you select a node template from a list are Export and Import. You can also perform these actions by using cmdlets in HPC PowerShell.

  • Additional information:

    Understanding Node Templates [Help link]

    Adding Nodes to a Cluster [Help link]

    Export-HpcNodeTemplate (https://go.microsoft.com/fwlink/?LinkId=182749)

    Import-HpcNodeTemplate (https://go.microsoft.com/fwlink/?LinkId=182837)

Node XML files

This section describes node XML files.

Note
Node XML files are different from node templates that are exported as XML files (described in the previous section).

  • Security considerations: Node XML files provide a flexible way to specify certain characteristics for nodes that otherwise would either be generated automatically or would have to be configured manually. For example, node XML files can specify custom computer names that are not automatically generated as a series. Protecting these files helps ensure that nodes have the characteristics that you intend, and helps prevent an attacker from connecting to and controlling compute nodes.

  • How node XML files are exported and imported: You can use HPC Cluster Manager to generate (export) a base node XML file, and then edit the file. Then you can use HPC Cluster Manager to add compute nodes by importing the edited node XML file. You can also perform these actions by using cmdlets in HPC PowerShell.

  • Additional information:

    Add Nodes by Importing a Node XML File

    Adding Nodes to a Cluster [Help link]

    Export-HpcNodeXML (https://go.microsoft.com/fwlink/?LinkId=182750

    Import-HpcNodeXML (https://go.microsoft.com/fwlink/?LinkId=182838)

Job and task XML files

This section describes job and task XML files.

Note
“Job and task XML files” are different from job templates that are exported as XML files (which are described in the next section).

  • Security considerations: When a job owner or HPC cluster administrator saves or exports settings for jobs and tasks, the resulting file is a job or task XML file. If someone tampers with a job or task XML file and the file is later used, it can cause the job or task to run in an unexpected way. If the file was deliberately damaged, the job or task might not run.

    Note
    While protecting job and task XML files, you (as an HPC administrator) can also limit or influence how jobs are run, by using job templates or job submission and activation filters. Job templates and filters are described later in this topic.

  • How job and task XML files are created and run: A job owner or HPC cluster administrator can use HPC Cluster Manager or HPC Job Manager to save (or export) job or task XML files. To do this, under Job Management, in the New Job properties sheet, click the Save Job XML File button (or on the Edit Tasks tab, the Save Task File button). For a defined job or task that is selected in a list, the Actions pane includes Export Job or Export Task. You can also perform these actions by using cmdlets in HPC PowerShell.

    After a job or task XML file is created, there are several interfaces that can be used for running a job with that file. The job can be run through HPC Cluster Manager or HPC Job Manager by selecting New Job from XML File and selecting the file. The job can also be run by using the New-HPCJob cmdlet with the -JobFile <string> parameter. (It can also be run by using job commands, although this is not mentioned in the following “Additional information” list).

  • Additional information:

    Save a Job or Task to a File [Admin]

    Export-HPCJob https://go.microsoft.com/fwlink/?LinkId=182745

    Export-HpcTask https://go.microsoft.com/fwlink/?LinkId=182751

    New-HPCJob (https://go.microsoft.com/fwlink/?LinkId=182842)

Job templates (exported as XML files)

This section describes job templates that are exported as XML files.

Note
Job templates that are exported as XML files are different from “job XML files” (which are described in the previous section).

  • Security considerations: A job template allows you to associate a set of default values and value constraints for job properties with a particular set of users. When you create job templates through HPC Cluster Manager, they are stored in one of the SQL Server databases that support the cluster, but they can also be exported into an XML file format. Protecting job templates (in any form) helps protect the cluster from denial-of-service (DoS) attacks or other attacks where jobs that use the templates are run in an unexpected way. It also helps control which users can run a particular job or set of jobs.

    For information about setting job template permissions, see the following “Additional information” list.

  • How job templates are exported or imported as XML: In HPC Cluster Manager, two of the actions that are available when you select a job template from a list are Export and Import. You can also perform these actions by using cmdlets in HPC PowerShell. When exporting a job template as an XML file, you specify the location and file name.

  • Additional information:

    Job Templates [Help link]

    Create a Job Template

    Set Job Template Permissions

    Export-HpcJobTemplate (https://go.microsoft.com/fwlink/?LinkId=182746)

    Import-HpcJobTemplate (https://go.microsoft.com/fwlink/?LinkId=182835)

Custom diagnostic tests (saved as XML files)

The following list describes custom diagnostic tests that are saved as XML files.

Files with a location that is determined during an installation or configuration process

For most of the files in the following list, the location is determined during an installation or configuration process, and no additional steps are needed to ensure that the HPC cluster can find the files. The exceptions are Excel add-ins (XLL files), which may include user-defined functions (UDFs). Because XLL files use a path variable, if you add XLL files to a custom location (for example, a shared folder on the network), you can then adjust the path variable so that the HPC cluster can locate the XLL files.

  • Job submission filters and job activation filters

  • Diagnostic test result files

  • SOA service registration files and the DLL files that they point to

  • Excel add-ins and any associated user-defined functions

Job submission filters and job activation filters

The following list describes job submission filters and job activation filters.

Diagnostic test result files

The following list describes diagnostic test result files.

  • Security considerations: Depending on the tests that have been run, diagnostic test result files might contain information about the cluster that could help an attacker determine where the cluster might be vulnerable.

  • Path: See Table of shared folders that are located on the head node, later in this topic.

    Note
    You can also use a cmdlet in HPC PowerShell to export the results of diagnostic tests, that is, to create a set of files that can be viewed later in HPC Cluster Manager. With exported test results, use the same level of security precautions as you would with any other stored test results.

  • Additional information:

    Diagnostics: HPC Cluster Manager [Help link]

    Export-HpcTestResult (https://go.microsoft.com/fwlink/?LinkId=182752)

SOA service registration files and the DLL files that they point to

The following list describes service-oriented architecture (SOA) service registration files and the DLL files that they point to.

  • Security considerations: The SOA service registration files are XML files that register SOA services on the cluster. The files also contain settings that configure service behavior and that point to the location of the DLL (assembly) for that service. Protecting the service registration files, and the associated DLL files, helps ensure that the services run as intended. Protecting these files can also help protect against the installation of malicious DLLs.

  • Path: The service registration files can be installed centrally (on the head node) or locally (on multiple nodes). For more information, see “Deploy and Edit the Service Configuration File” in the following “Additional information” list.

    For information about where files are located if you have installed them centrally on the head node, see Table of shared folders that are located on the head node, later in this topic.

  • Additional information:

    SOA Service Configuration Files in Windows HPC Server 2008 R2 (https://go.microsoft.com/fwlink/?LinkId=208482)

    Deploy and Edit the Service Configuration File

Excel add-ins and any associated user-defined functions

The following list describes Excel add-ins (XLL files), which may include user-defined functions (UDFs).

Table of shared folders that are located on the head node

By default, the following types of files are placed in shared folders on the head node. These shared folders provide network access to information that is stored on the head node.

Files in shared folder Name of shared folder Example location for head node installed in default location on drive C

Files needed for installation of compute nodes, for example, images, driver files, and custom Diskpart.txt files

\\<headnode>\REMINST

This folder has subfolders. One example is the Image subfolder.

C:\Program Files\Microsoft HPC Pack 2008 R2\Data\InstallShare

This folder has subfolders. One example is the Image subfolder.

Diagnostic test results

\\<headnode>\Diagnostics

C:\Program Files\Microsoft HPC Pack 2008 R2\Data\Diagnostics

SOA service registration files

\\<headnode>\HPCServiceRegistration

C:\Program Files\Microsoft HPC Pack 2008 R2\ServiceRegistration

Trace files or debugging output files from the cluster

\\<headnode>\CcpSpoolDir

C:\Program Files\Microsoft HPC Pack 2008 R2\Data\SpoolDir

Files in temporary folders on workstation nodes

If your HPC cluster includes workstation nodes, your security planning should include a review of the location of job input and output folders on each workstation node, and how those folders (and the files they contain) are protected. The location of these folders can vary, depending on the application. For example, an application may create a folder called c:\Temp\Output and store files in it. Review the following considerations for these folders:

  • Limit permissions on these folders, so that unintended users cannot access them. This can help keep such users from tampering with files in those folders (which could have an effect on how jobs run) or viewing sensitive information in those files.

  • Consider placing quotas on these folders to limit folder size. This can help prevent a denial-of-service attack that uses jobs to write large quantities of data to these folders, filling up the disks on the workstations and causing them to become unusable.

Also, it can be important to review the software or scripts that run jobs on workstation nodes, to ensure that data is not left behind in input or output folders after the script or software runs. This can help prevent unintended access to that data, and help prevent the disk from filling up with data mistakenly left behind by past jobs.

Caution
We strongly recommend that you do not add workstation computers to your HPC cluster on which users have local administrative privileges. For more information, see "Considerations for user accounts used on the workstation computers" in Requirements for Adding Workstation Nodes in Windows HPC Server 2008 R2 (https://go.microsoft.com/fwlink/?LinkId=202684).

Specialized files for HPC session configuration, power control, and unattended installation

An HPC cluster includes a small number of specialized files for HPC session configuration, power control, and unattended installation. These are powerful files, and can be customized. If you customize these files, you should track and handle them carefully, and protect the folders where you store them. For example, you should limit access to a temporary folder where you are modifying a copy of the unattended installation file called Unattend.xml. If an attacker makes changes to the file without your awareness, you might later deploy that version of the file, which would result in unintended changes to the way that the compute nodes are installed.

These specialized files are described in the following list. They are all installed on the head node in the \Bin subfolder. For example, if you are using the default installation path, the files are available in the following folder:

C:\Program Files\Microsoft HPC Pack 2008 R2\Bin

The files are as follows:

  • HpcSession.exe.config is a configuration file that you can use to customize your HPC configuration, for example, to provide the head node with the name of Windows Communication Foundation (WCF) broker node that is running on a failover cluster instead of running as a stand-alone server.

  • CcpPower.cmd is the centralized power control script for starting and stopping compute nodes. A default version of the script is installed with HPC Pack 2008 R2. The default version calls standard operating system commands to stop and restart compute nodes. However, you can modify CcpPower.cmd so that it calls custom power control scripts that control the way that compute nodes are started, stopped, or restarted. An example of a custom power control script is an Intelligent Platform Management Interface (IPMI) script.

    Note
    It is important to track and protect not only CcpPower.cmd, but also the custom scripts that it calls. This helps ensure that compute nodes will start, stop, and restart as expected. It also helps protect against installation of malicious software (malware) or other unintended software on the compute nodes.
    For additional informational information about **CcpPower.cmd**, see [Appendix 5: Scriptable Power Control Tools](https://go.microsoft.com/fwlink/?linkid=202856) (https://go.microsoft.com/fwlink/?LinkId=202856).

  • Startnet.cmd and Unattend.xml are files that are associated with the Pre-Boot Execution Environment (PXE) and unattended installation. You can use these files to customize the process of unattended installation for your compute nodes.

    Other files related to unattended installation are images, driver files, and custom disk-partitioning files (Diskpart.txt files). For more information about these files, see Node templates (exported as XML files) and Table of shared folders that are located on the head node, earlier in this topic.

Log files that contain detailed information such as trace information from SOA sessions

In Windows HPC Server 2008 R2, you can collect detailed information in log files and use various methods to copy these log files to other locations. For example, you can collect diagnostic traces for SOA sessions and save the traces in WCF log files. WCF log files can then be copied (by using HPC Cluster Manager or a cmdlet in HPC PowerShell) from the compute nodes to a single location so that you can view the log files with Service Trace Viewer.

Because detailed log files (WCF log files or other detailed log files) can contain important information such as information about the HPC cluster topology and nodes, carefully track and protect the locations where you save them. Also, if you collect traces or other detailed logs, we recommend that you remove these files after you are finished reviewing them. For additional information about log files, see the following topics:

Additional references