Creating and Submitting Jobs by Using the REST API in Microsoft HPC Pack (Windows HPC Server)

Microsoft HPC Pack (Windows HPC Server) provides programmatic access to the HPC Job Scheduler Service through an HTTP web service that is based on the representational state transfer (REST) model, starting with Microsoft HPC Pack 2008 R2 with Service Pack 2 (SP2). You can use this REST API to create client applications that users can use to define, submit, modify, list, view, requeue, and cancel jobs. Later versions of HPC Pack, including HPC Pack 2012, allow you to use the REST API to get information about nodes, node groups, and the cluster name, as well as create and manage SOA sessions, and to send SOA requests and receive SOA responses for these sessions. For more information and prerequisites to use the REST API, see Working with the Web Service Interface (http://msdn.microsoft.com/en-us/library/hh560254(VS.85).aspx).

You can use the REST API in many programming languages to create client applications, including programming languages that the .NET and COM APIs for Windows HPC Server do not support. The HPC REST API defines a number of operations that you can perform, including the information that is expected in the HTTP requests that you send to the HPC REST web service and that is provided in the HTTP responses that you receive in return as part of each operation. The HPC REST API does not define the manner in which you create and send these HTTP requests and process the HTTP responses. The approach you can use to work with the HTTP requests and responses varies depending on the programming language that you use, and can also vary depending on the libraries or other tools that you choose to install to work with HTTP requests and responses in your programming language of choice.

This article provides examples of how to work with the HTTP requests and responses in the C# programming language for the operations that support a common procedure in Windows HPC Server, creating and submitting a job. The examples use items in the System.Net namespace of the .NET Framework class library to work with HTTP requests and responses, particularly the System.Net.HttpWebRequest and System.Net.HttpWebResponse classes. This approach for working with the HTTP requests and responses in the operations of the REST API is also the approach used by the C# example in the REST API code samples starting with the Microsoft HPC Pack 2008 R2 SDK with SP3. Members of the Windows HPC Server community are encouraged to add their own examples for the programming languages and libraries of their choice.

Working with REST API Operations

An operation in the HPC REST API consists of an HTTP request that you send to the REST web service and the HTTP response that you receive in return. For information about the operations available in the REST API, see HPC Web Service API Reference (http://msdn.microsoft.com/en-us/library/hh560258(VS.85).aspx).

An HTTP request can include the following information:

• A URI
• A method
• Credentials
• Headers
• URI parameters
• The request body

An HTTP response can include the following information:

• A status code
• Headers
• The response body

If you use the .NET Framework for programming your applications, you can use the methods and properties of the System.Net.HttpWebRequest and System.Net.HttpWebResponse classes to set or get the items that are part of these HTTP requests and responses.

Creating and Submitting Jobs in a C# Application by Using the REST API

One common procedure that users of Windows HPC Server perform is to create and submit a job. This procedure includes multiple high-level tasks, each of which corresponds to an operation in the HPC REST API. Performing each of these operations within a console application consists of several steps.

To create and submit a job by using the REST API

1. Prepare to perform REST API operations by adding a set of using directives, declaring variables to hold job and task information, turning off the enforcement of certificate trust chains, and prompting the user for credentials to use to connect to the REST web service and run jobs.

2. Create a new job with the Create Job operation.

3. Add a task to the job with the Add Task operation.

4. Submit the job to the HPC Job Scheduler Service so that it can run on the HPC Cluster by using the Submit Job operation.

5. Check periodically whether the job has finished by using the Get Job operation to check the value of the State property of the job.

6. Get the output of the tasks in the job once the job finishes by using the Get Task operation to view the value of the Output property of the task.

The sections later in this article that describe each of these tasks focus on creating the code to perform the steps in those tasks. To see the code for these steps in the context of a complete example, see the Complete Code Example later in this article.

Step 1: Prepare to Perform REST API Operations

To prepare to perform REST API operations in your application, you need to do the following:

• Include using directives for the namespaces that contain the types that you want to use in your application, including the System.Net namespace that contains the types for working with HTTP requests and responses.
• Declare variables to hold information about jobs and tasks between operations.
• Turn off the enforcement of certificate trust chains, if the certificate for the REST web service to which you want to connect uses a self-signed certificate.
• Prompt the user for the credentials of an account that can connect to the REST web service and submit jobs to run on the HPC cluster.

To prepare to perform REST API operations

1. Change the set of using directives at the beginning of the file that contains your code so that they include the namespaces for the types that you want to use in your application. In particular, include a using statement for the System.Net namespace that contains the types for working with HTTP requests and responses. For example:

   using System;
   using System.Collections.Generic;
   using System.IO;
   using System.Net;
   using System.Net.Security;
   using System.Runtime.Serialization;
   using System.Security.Cryptography.X509Certificates;
   using System.Text;
   using System.Threading;
   

2. Inside the Program class for the console application, but outside the Main method for the class, declare variables to hold the values of the job identifier, task identifier, and job state between REST API operations. For example:

   class Program
     {
   
       static int jobId;
       static int taskId;
       static string jobState;
   
       static void Main(string[] args)
         {
   

3. Inside the Main method for your console application, turn off the enforcement of certificate trust chains by setting a callback function for validating server certificates, if the certificate for the REST web service to which you want to connect uses a self-signed certificate. For example:

   ServicePointManager.ServerCertificateValidationCallback = new System.Net.Security.RemoteCertificateValidationCallback(DisableTrustChainEnforecment);
   

   If you want to connect to a REST web service that uses a certificate that is not a self-signed certificate, leave out this line of code or place it inside a comment to enforce trust chains between your REST client application and the REST web service.

   For a definition of the DisableTrustChainEnforecment callback function, see "Complete Code Example" later in this article.

4. Prompt the user for the user name and password of an account that can connect to the REST web service and submit jobs to run on the HPC cluster, and create an System.Net.ICredentials interface for these credentials. For example:

   string credUserName;
   Console.Write("User name to run job: ");
   credUserName = Console.ReadLine();
   
   string credPassword = string.Empty;
   Console.Write("Password to run job: ");
   
   ConsoleKeyInfo keyInfo = Console.ReadKey(true);
   
   while (keyInfo.Key != ConsoleKey.Enter)
     {
   
       Console.Write("*");
       credPassword += keyInfo.KeyChar;
       keyInfo = Console.ReadKey(true);
   
     }
   
   Console.WriteLine();
   
   // Create credentials to use for basic authentication.
   ICredentials credsBasic = new NetworkCredential(credUserName, credPassword);
   

Step 2: Create a New Job

To create a new job with the REST API, you can use the Create Job or Create Job from XML operation. In the Create Job operation, the XML that you send in the HTTP request body is similar to the XML in the request body for other HPC REST API operations. In the Create Job from XML operation, you can use the same job XML format for the HTTP request body that you use in a job XML file for the /jobfile parameter of the job new command or the -JobFile parameter of the New-HpcJob cmdlet. This example uses the Create Job operation.

To create a job

1. Create an HTTP request for the Create Job operation by calling the WebRequest.Create method that the System.Net.HttpWebRequest class inherits, and specify the URI by using a format of https://head_node_name[:port]/WindowsHpc/HPC_cluster_name/Jobs. For example:

   HttpWebRequest req = HttpWebRequest.Create("https://myheadnode.contoso.com/WindowsHpc/myheadnode/Jobs") as HttpWebRequest;
   

2. Set the credentials for the HTTP request by setting the HttpWebRequest.Credentials property. For example:

   req.Credentials = credsBasic;
   

3. Set the values for the api-version and Content-Type headers for the HTTP request. To set the api-version header, use the HttpWebRequest.Headers property. To set the Content-Type header, set the value of the HttpWebRequest.ContentType property. For example:

   req.Headers["api-version"] = "2011-11-01";
   req.ContentType = "application/xml; charset=utf-8";
   

4. Set the method for the HTTP request by setting HttpWebRequest.Method property to POST. For example:

   req.Method = "POST";
   

5. Specify the request body by writing the data to a stream for the request. For this example, this data is provided as an array of bytes in ASCII encoding that represents an XML string in the format that the Create Job operation expects. To specify the request body in this manner:

   1. Create a new System.Text.ASCIIEncoding object and use the ASCIIEncoding.GetBytes method to get an array of bytes for the XML string that you want to include in the request. In this example, the XML specifies the value of the ExpandedPriority property for the job as 3250.

      ASCIIEncoding encoding = new ASCIIEncoding();
      byte[] byte1 = encoding.GetBytes("<ArrayOfProperty xmlns=\"http://schemas.microsoft.com/HPCS2008R2/common\"><Property><Name>ExpandedPriority</Name><Value>3250</Value></Property></ArrayOfProperty>");
      

   2. Call the HttpWebRequest.GetRequestStream method to get a request stream. For example:

      Stream reqStream = req.GetRequestStream();
      

   3. Call the Stream.Write method to write the array of bytes to the request stream, and then call the Stream.Close method when you are done. For example:

      reqStream.Write(byte1, 0, byte1.Length);
      reqStream.Close();
      

6. Get the HTTP response for the request by calling the HttpWebRequest.GetResponse method. For example:

   HttpWebResponse response = req.GetResponse() as HttpWebResponse;
   

7. Get the status code for the HTTP response by getting the value of the HttpWebResponse.StatusCode property, and if the status code is not OK, call a function that prints information about the error and exits. For example:

   HttpStatusCode statusCode = response.StatusCode;
   
   if (statusCode != HttpStatusCode.OK)
   {
       HandleHttpErrors(response);
   }
   

   To see the definition of the HandleHttpErrors method, see "Complete Code Example" later in this article.

8. Process the response body by getting the response stream, reading the stream, and getting the substring from the response body that represents the job identifier for the newly created job. To process the response body in this manner:

   1. Get the response stream for the response by calling the HttpWebResponse.GetResponseStream method. For example:

      Stream respStream = response.GetResponseStream();
      

   2. Create a System.IO.StreamReader object for the stream, then use the StreamReader.ReadToEnd method to read the stream and save the response body in a string. For example:

      using (StreamReader reader = new StreamReader(respStream))
        {
      
          string retString = reader.ReadToEnd();
      

   3. Extract the job identifier from the response body by using the String.Substring and String.IndexOf methods to get the portion of the XML response that represents the job identifier, and use the Convert.ToInt32 method convert the substring to an integer. For example:

          jobId = Convert.ToInt32(retString.Substring(retString.IndexOf("<int ") + 65, retString.IndexOf("</int>") - retString.IndexOf("<int ") - 65));
      

      The format of the response body for the Create Job operation is <int xmlns="http://schemas.microsoft.com/2003/10/Serialization/">job_identifier</int>.

   4. Close the System.IO.StreamReader object by calling the StreamReader.Close and Stream.Close methods.

          reader.Close();
          respStream.Close();
      
        }
      

9. Finish the work for this operation by calling the Console.WriteLine method to display the job identifier of the newly created job to user, and by calling the HttpWebResponse.Close method to close the response.

   Console.WriteLine("Created job {0}.", jobId);
   response.Close();
   

Step 3: Add a Task to the Job

To add a new task to a job by using the REST API, you can use the Add Task operation. The steps for performing this operation are similar to those you used to perform the Create Job operation.

To add a task to the job

1. Create an HTTP request for the Add Task operation by calling the WebRequest.Create method that the System.Net.HttpWebRequest class inherits, and specify the URI by using a format of https://head_node_name[:port]/WindowsHpc/HPC_cluster_name/Job/job_identifier/Tasks. For example:

   HttpWebRequest req2 = HttpWebRequest.Create("https://myheadnode.contoso.com/WindowsHpc/myheadnode/Job/" + jobId + "/Tasks") as HttpWebRequest;
   

2. Set the credentials for the HTTP request by setting the HttpWebRequest.Credentials property. For example:

   req2.Credentials = credsBasic;
   

3. Set the values for the api-version and Content-Type headers for the HTTP request. To set the api-version header, use the HttpWebRequest.Headers property. To set the Content-Type header, set the value of the HttpWebRequest.ContentType property. For example:

   req2.Headers["api-version"] = "2011-11-01";
   req2.ContentType = "application/xml; charset=utf-8";
   

4. Set the method for the HTTP request by setting HttpWebRequest.Method property to POST. For example:

   req2.Method = "POST";
   

5. Specify the request body by writing the data to a stream for the request. For this example, this data is provided as an array of bytes in ASCII encoding that represents an XML string in the format that the Add Task operation expects. To specify the request body in this manner:

   1. Create a new System.Text.ASCIIEncoding object and use the ASCIIEncoding.GetBytes method to get an array of bytes for the XML string that you want to include in the request. In this example, the XML specifies the command that the task should run by specifying a value of echo Hello World! for the CommandLine property of the task.

      ASCIIEncoding encoding2 = new ASCIIEncoding();
      byte[] byte2 = encoding2.GetBytes("<ArrayOfProperty xmlns=\"http://schemas.microsoft.com/HPCS2008R2/common\"><Property><Name>CommandLine</Name><Value>echo Hello World!</Value></Property></ArrayOfProperty>");
      

   2. Call the HttpWebRequest.GetRequestStream method to get a request stream. For example:

      Stream reqStream2 = req2.GetRequestStream();
      

   3. Call the Stream.Write method to write the array of bytes to the request stream, and then call the Stream.Close method when you are done. For example:

      reqStream2.Write(byte2, 0, byte2.Length);
      reqStream2.Close();
      

6. Get the HTTP response for the request by calling the HttpWebRequest.GetResponse method. For example:

   HttpWebResponse response2 = req2.GetResponse() as HttpWebResponse;
   

7. Get the status code for the HTTP response by getting the value of the HttpWebResponse.StatusCode property, and if the status code is not OK, call a function that prints information about the error and exits. For example:

   HttpStatusCode statusCode2 = response2.StatusCode;
   
   if (statusCode2 != HttpStatusCode.OK)
   {
       HandleHttpErrors(response2);
   }
   

   To see the definition of the HandleHttpErrors method, see "Complete Code Example" later in this article.

8. Process the response body by getting the response stream, reading the stream, and getting the substring from the response body that represents the task identifier for the newly added task. To process the response body in this manner:

   1. Get the response stream for the response by calling the HttpWebResponse.GetResponseStream method. For example:

      Stream respStream2 = response2.GetResponseStream();
      

   2. Create a System.IO.StreamReader object for the stream, then use the StreamReader.ReadToEnd method to read the stream and save the response body in a string. For example:

      using (StreamReader reader2 = new StreamReader(respStream2))
        {
      
          string retString2 = reader2.ReadToEnd();
      

   3. Extract the task identifier from the response body by using the String.Substring and String.IndexOf methods to get the portion of the XML response that represents the task identifier, and use the Convert.ToInt32 method convert the substring to an integer. For example:

          taskId = Convert.ToInt32(retString2.Substring(retString2.IndexOf("<int ") + 65, retString2.IndexOf("</int>") - retString2.IndexOf("<int ") - 65));
      

      The format of the response body for the Add Task operation is <int xmlns="http://schemas.microsoft.com/2003/10/Serialization/">task_identifier</int>.

   4. Close the System.IO.StreamReader object by calling the StreamReader.Close and Stream.Close methods.

          reader2.Close();
          respStream2.Close();
      
        }
      

9. Finish the work for this operation by calling the Console.WriteLine method to display the task identifier of the newly added task to user, and by calling the HttpWebResponse.Close method to close the response.

   Console.WriteLine("Added task {0} to job {1}.", taskId, jobId);
   response2.Close();
   

Step 4: Submit the Job

To submit a job by using the REST API, you can use the Submit Job operation. The Submit Job operation differs from the Create Job and Add Task operations in that the body of the response for the Submit Job operation is empty, so the code that processes the response just checks the status code and handles any errors.

To submit the job

1. Create an HTTP request for the Submit Job operation by calling the WebRequest.Create method that the System.Net.HttpWebRequest class inherits, and specify the URI by using a format of https://head_node_name[:port]/WindowsHpc/HPC_cluster_name/Job/job_identifier/Submit. For example:

   HttpWebRequest req3 = HttpWebRequest.Create("https://myheadnode.contoso.com/WindowsHpc/myheadnode/Job/" + jobId + "/Submit") as HttpWebRequest;
   

2. Set the credentials for the HTTP request by setting the HttpWebRequest.Credentials property. For example:

   req3.Credentials = credsBasic;
   

3. Set the values for the api-version and Content-Type headers for the HTTP request. To set the api-version header, use the HttpWebRequest.Headers property. To set the Content-Type header, set the value of the HttpWebRequest.ContentType property. For example:

   req3.Headers["api-version"] = "2011-11-01";
   req3.ContentType = "application/xml; charset=utf-8";
   

4. Set the method for the HTTP request by setting HttpWebRequest.Method property to POST. For example:

   req3.Method = "POST";
   

5. Specify the request body by writing the data to a stream for the request. For this example, this data is provided as an array of bytes in ASCII encoding that represents an XML string in the format that the Submit Job operation expects. To specify the request body in this manner:

   1. Create a new System.Text.ASCIIEncoding object and use the ASCIIEncoding.GetBytes method to get an array of bytes for the XML string that you want to include in the request. In this example, the XML specifies that you want to change the value of the Name property of the job to My Job.

      ASCIIEncoding encoding3 = new ASCIIEncoding();
      byte[] byte3 = encoding3.GetBytes("<ArrayOfProperty xmlns=\"http://schemas.microsoft.com/HPCS2008R2/common\"><Property><Name>Name</Name><Value>My Job</Value></Property></ArrayOfProperty>");
      

   2. Call the HttpWebRequest.GetRequestStream method to get a request stream. For example:

      Stream reqStream3 = req3.GetRequestStream();
      

   3. Call the Stream.Write method to write the array of bytes to the request stream, and then call the Stream.Close method when you are done. For example:

      reqStream3.Write(byte3, 0, byte3.Length);
      reqStream3.Close();
      

6. Get the HTTP response for the request by calling the HttpWebRequest.GetResponse method. For example:

   HttpWebResponse response3 = req3.GetResponse() as HttpWebResponse;
   

7. Get the status code for the HTTP response by getting the value of the HttpWebResponse.StatusCode property, and if the status code is not OK, call a function that prints information about the error and exits. For example:

   HttpStatusCode statusCode3 = response3.StatusCode;
   
   if (statusCode3 != HttpStatusCode.OK)
   {
       HandleHttpErrors(response3);
   }
   

   To see the definition of the HandleHttpErrors method, see "Complete Code Example" later in this article.

8. Finish the work for this operation by calling the Console.WriteLine method to display a message to user that confirms that the job was submitted.

   Console.WriteLine("Job {0} submitted.", jobId);
   

Step 5: Wait until the Job Finishes

To determine if the job is finished, you need to check the value of the State property of the job. Because a job can take some time to run and may also need to wait for other jobs to finish before the job can run, you may need to check the State property multiple times, and need to wait for some time between successive attempts to check the value of the State property. So, you need to put the code that checks the value of the State property in a loop, and exit that loop when the State property indicates that the job is finished.

You can check the values of the properties of jobs by using the Get Job operation of the REST API. You can specify the job properties for which you want the values by specifying them as values for the Properties URI parameter. The Get Job operation also differs from the other operations that you performed previously in this scenario in that the body of the request for the Get Job operation is empty, so the code does not write data to the request stream for this operation.

To wait until the job finishes

1. Start a loop that checks if the job is finished and then exits when the job is finished by including a do statement, then call the Thread.Sleep method to pause the application before each time the application checks the job state in order to allow time for the job to run and finish.

   do
     {
        Thread.Sleep(2000);
   

2. Create an HTTP request for the Get Job operation by calling the WebRequest.Create method that the System.Net.HttpWebRequest class inherits, and specify the URI by using a format of https://head_node_name[:port]/WindowsHpc/HPC_cluster_name/Job/job_identifier?Properties=State. This URI includes the Properties URI parameter to specify that you only want to get the value of the State property of the job. For example:

   HttpWebRequest req4 = HttpWebRequest.Create("https://myheadnode.contoso.com/WindowsHpc/myheadnode/Job/" + jobId + "?Properties=State") as HttpWebRequest;
   

3. Set the credentials for the HTTP request by setting the HttpWebRequest.Credentials property. For example:

   req4.Credentials = credsBasic;
   

4. Set the values for the api-version header for the HTTP request by setting the HttpWebRequest.Headers property. For example:

   req4.Headers["api-version"] = "2011-11-01";
   

5. Set the method for the HTTP request by setting HttpWebRequest.Method property to GET. For example:

   req4.Method = "GET";
   

6. Get the HTTP response for the request by calling the HttpWebRequest.GetResponse method. For example:

   HttpWebResponse response4 = req4.GetResponse() as HttpWebResponse;
   

7. Get the status code for the HTTP response by getting the value of the HttpWebResponse.StatusCode property, and if the status code is not OK, call a function that prints information about the error and exits. For example:

   HttpStatusCode statusCode4 = response4.StatusCode;
   
   if (statusCode4 != HttpStatusCode.OK)
   {
       HandleHttpErrors(response4);
   }
   

   To see the definition of the HandleHttpErrors method, see "Complete Code Example" later in this article.

8. Process the response body by getting the response stream, reading the stream, and getting the substring from the response body that indicates the state of the job. To process the response body in this manner:

   1. Get the response stream for the response by calling the HttpWebResponse.GetResponseStream method. For example:

      Stream respStream4 = response4.GetResponseStream();
      

   2. Create a System.IO.StreamReader object for the stream, then use the StreamReader.ReadToEnd method to read the stream and save the response body in a string. For example:

      using (StreamReader reader4 = new StreamReader(respStream4))
        {
      
          string retString4 = reader4.ReadToEnd();
      

   3. Extract the job state from the response body by using the String.Substring and String.IndexOf methods to get the portion of the XML response that represents the job state. For example:

          string outputString = retString4.Substring(retString4.IndexOf("<Value>") + 7, retString4.IndexOf("</Value>") - retString4.IndexOf("<Value>") - 7);
      

      The format of the response body for this Get Job operation is <ArrayOfProperty xmlns="http://schemas.microsoft.com/HPCS2008R2/common" xmlns:i="http://www.w3.org/2001/XMLSchema-instance"><Property><Name>State</Name><Value>job_state</Value></Property></ArrayOfProperty>. The XML response in this example includes only one set of Property, Name, and Value elements because you specified a single property for the Properties URI parameter. This set of elements provides the value of the State property of the job. Because the XML only contains one Value element, the values specified for the parameters of String.Substring are sufficiently specific to extract the value of the State property.

   4. Close the System.IO.StreamReader object by calling the StreamReader.Close and Stream.Close methods.

          reader4.Close();
          respStream4.Close();
      
        }
      

   5. Finish the work for this operation by calling the HttpWebResponse.Close method to close the response.

      response4.Close();
      

   6. Check if the job state is Failed or Canceled. If the state of the job is Failed or Canceled, call the Console.WriteLine method to display an error message to the user, and then call the Environment.Exit method to exit your application. You only want to continue with the last task in the program of getting the output of the job if the job finishes successfully. For example:

      if (jobState == "Failed" || jobState == "Canceled")
        {
          Console.WriteLine("Error: Job {0} completed with a state of {1}.", jobId, jobState);
          Environment.Exit(-1);
        }
      

   7. Close the loop with a while statement that checks if the state of the job is Finished. If the job state is Finished, you can proceed to the final task of getting the output of the task in the job.

        } while (jobState != "Finished" && jobState != "Failed" && jobState != "Canceled");
      

Step 6: Get the Output of the Tasks in the Job

To get the output for a task in a job once the job is finished, you can check value of the Output property of the task.

Note

The Output property of the task contains the result for the task only if the command for the task wrote the result to standard output. If the command that you run for the task writes its results to one or more files, check with your cluster administrator to determine where your command should save the output files and how you can retrieve those files. You need to process those output files as appropriate for their file type to provide the results to the user, rather than using the Output property of the task.

You can check the values of the properties of a task by using the Get Task operation. You can specify the task properties for which you want the values by specifying them as values for the Properties URI parameter. The Get Task operation is similar to the Get Job operation in that the body of the request for the Get Task operation is empty, so the code does not write data to the request stream for this operation.

To get the output for the task in the job

1. Create an HTTP request for the Get Task operation by calling the WebRequest.Create method that the System.Net.HttpWebRequest class inherits, and specify the URI by using a format of https://head_node_name[:port]/WindowsHpc/HPC_cluster_name/Job/job_identifier/Task/task_identifier?Properties=Output. This URI includes the Properties URI parameter to specify that you only want to get the value of the Output property of the task. For example:

   HttpWebRequest req5 = HttpWebRequest.Create("https://myheadnode.contoso.com/WindowsHpc/myheadnode/Job/" + jobId + "/Task/" + taskId + "?Properties=Output") as HttpWebRequest;
   

2. Set the credentials for the HTTP request by setting the HttpWebRequest.Credentials property. For example:

   req5.Credentials = credsBasic;
   

3. Set the values for the api-version header for the HTTP request by setting the HttpWebRequest.Headers property. For example:

   req5.Headers["api-version"] = "2011-11-01";
   

4. Set the method for the HTTP request by setting HttpWebRequest.Method property to GET. For example:

   req5.Method = "GET";
   

5. Get the HTTP response for the request by calling the HttpWebRequest.GetResponse method. For example:

   HttpWebResponse response5 = req5.GetResponse() as HttpWebResponse;
   

6. Get the status code for the HTTP response by getting the value of the HttpWebResponse.StatusCode property, and if the status code is not OK, call a function that prints information about the error and exits. For example:

   HttpStatusCode statusCode5 = response5.StatusCode;
   
   if (statusCode5 != HttpStatusCode.OK)
   {
       HandleHttpErrors(response5);
   }
   

   To see the definition of the HandleHttpErrors method, see "Complete Code Example" later in this article.

7. Process the response body by getting the response stream, reading the stream, and getting the substring from the response body that indicates the output of the task. To process the response body in this manner:

   1. Get the response stream for the response by calling the HttpWebResponse.GetResponseStream method. For example:

      Stream respStream5 = response5.GetResponseStream();
      

   2. Create a System.IO.StreamReader object for the stream, then use the StreamReader.ReadToEnd method to read the stream and save the response body in a string. For example:

      using (StreamReader reader5 = new StreamReader(respStream5))
        {
      
          string retString5 = reader5.ReadToEnd();
      

   3. Extract the task output from the response body by using the String.Substring and String.IndexOf methods to get the portion of the XML response that represents the task output. For example:

          string outputString = retString5.Substring(retString5.IndexOf("<Value>") + 7, retString5.IndexOf("</Value>") - retString5.IndexOf("<Value>") - 7);
      

      The format of the response body for this Get Task operation is <ArrayOfProperty xmlns="http://schemas.microsoft.com/HPCS2008R2/common" xmlns:i="http://www.w3.org/2001/XMLSchema-instance"><Property><Name>Output</Name><Value>task_output</Value></Property></ArrayOfProperty>. The XML response in this example includes only one set of Property, Name, and Value elements because you specified a single property for the Properties URI parameter. This set of elements provides the value of the Output property of the task. Because the XML only contains one Value element, the values specified for the parameters of String.Substring are sufficiently specific to extract the value of the Output property.

   4. Close the System.IO.StreamReader object by calling the StreamReader.Close and Stream.Close methods.

          reader5.Close();
          respStream5.Close();
      
        }
      

8. Finish the work for this operation by calling the HttpWebResponse.Close method to close the response.

   response5.Close();
   }
   

   When you complete the code for the Get Task operation, the code needed for the Main method of the class for your console application is complete.

Complete Code Example

The following example shows a complete code file that includes the examples earlier in this article in context, including additional helper functions and classes to support those examples.

using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Net.Security;
using System.Runtime.Serialization;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using System.Threading;

namespace JobExample
{
    class Program
    {

        // Declare variables to hold job and task information between operations.
        static int jobId;
        static int taskId;
        static string jobState;

        static void Main(string[] args)
        {

            // This line turns off enforcement of certificate trust chains, which
            // enables the use of self-signed certificates.
            // Comment out this line to enforce trusted chains between your REST service and
            // REST clients.
            ServicePointManager.ServerCertificateValidationCallback = new System.Net.Security.RemoteCertificateValidationCallback(DisableTrustChainEnforecment);

            // Get the credentials from the user.
            string credUserName;
            Console.Write("User name to run job: ");
            credUserName = Console.ReadLine();

            string credPassword = string.Empty;
            Console.Write("Password to run job: ");

            ConsoleKeyInfo keyInfo = Console.ReadKey(true);

            while (keyInfo.Key != ConsoleKey.Enter)
            {

                Console.Write("*");
                credPassword += keyInfo.KeyChar;
                keyInfo = Console.ReadKey(true);

            }

            Console.WriteLine();

            // Create credentials to use for basic authentication.
            ICredentials credsBasic = new NetworkCredential(credUserName, credPassword);

            // Create a new job with the Create Job operation.
            try
            {

                HttpWebRequest req = HttpWebRequest.Create("https://myheadnode.contoso.com/WindowsHpc/myheadnode/Jobs") as HttpWebRequest;

                req.Credentials = credsBasic;

                // Add request headers.
                req.Headers["api-version"] = "2011-11-01";
                req.ContentType = "application/xml; charset=utf-8";
                req.Method = "POST";

                // Write the request body to the request stream.
                ASCIIEncoding encoding = new ASCIIEncoding();
                byte[] byte1 = encoding.GetBytes("<ArrayOfProperty xmlns=\"http://schemas.microsoft.com/HPCS2008R2/common\"><Property><Name>ExpandedPriority</Name><Value>3250</Value></Property></ArrayOfProperty>");

                Stream reqStream = req.GetRequestStream();

                reqStream.Write(byte1, 0, byte1.Length);

                reqStream.Close();

                HttpWebResponse response = req.GetResponse() as HttpWebResponse;

                HttpStatusCode statusCode = response.StatusCode;

                if (statusCode != HttpStatusCode.OK)
                {
                    HandleHttpErrors(response);
                }

                // Get the job identifier from the response body.
                Stream respStream = response.GetResponseStream();

                using (StreamReader reader = new StreamReader(respStream))
                {
                    string retString = reader.ReadToEnd();

                    jobId = Convert.ToInt32(retString.Substring(retString.IndexOf("<int ") + 65, retString.IndexOf("</int>") - retString.IndexOf("<int ") - 65)); 
                    
                    reader.Close();
                    respStream.Close();

                }

                Console.WriteLine("Created job {0}.", jobId); 
                response.Close();

            }

            catch (Exception ex)
            {
                Console.WriteLine("Exception: " + ex.ToString());

                HandleException(ex);
                throw;
            }

            // Add a task to the job with the Add Task operation.
            try
            {

                HttpWebRequest req2 = HttpWebRequest.Create("https://myheadnode.contoso.com/WindowsHpc/myheadnode/Job/" + jobId + "/Tasks") as HttpWebRequest;

                req2.Credentials = credsBasic;

                // Add request headers.
                req2.Headers["api-version"] = "2011-11-01";
                req2.ContentType = "application/xml; charset=utf-8";
                req2.Method = "POST";

                // Write the request body to the request stream.
                ASCIIEncoding encoding2 = new ASCIIEncoding();
                byte[] byte2 = encoding2.GetBytes("<ArrayOfProperty xmlns=\"http://schemas.microsoft.com/HPCS2008R2/common\"><Property><Name>CommandLine</Name><Value>echo Hello World!</Value></Property></ArrayOfProperty>");

                Stream reqStream2 = req2.GetRequestStream();

                reqStream2.Write(byte2, 0, byte2.Length);

                reqStream2.Close();

                HttpWebResponse response2 = req2.GetResponse() as HttpWebResponse;

                HttpStatusCode statusCode2 = response2.StatusCode; 
                if (statusCode2 != HttpStatusCode.OK)
                {
                    HandleHttpErrors(response2);
                }

                // Get the task identifier from the response body.
                Stream respStream2 = response2.GetResponseStream();

                using (StreamReader reader2 = new StreamReader(respStream2))
                {
                    string retString2 = reader2.ReadToEnd();

                    taskId = Convert.ToInt32(retString2.Substring(retString2.IndexOf("<int ") + 65, retString2.IndexOf("</int>") - retString2.IndexOf("<int ") - 65));

                    reader2.Close();
                    respStream2.Close();

                } 

                Console.WriteLine("Added task {0} to job {1}.", taskId, jobId);
                response2.Close();

            }

            catch (Exception ex2)
            {
                Console.WriteLine("Exception: " + ex2.ToString());

                HandleException(ex2);
                throw;
            }

            // Submit the job with the Submit Job operation.
            try
            {

                HttpWebRequest req3 = HttpWebRequest.Create("https://myheadnode.contoso.com/WindowsHpc/myheadnode/Job/" + jobId + "/Submit") as HttpWebRequest;

                req3.Credentials = credsBasic;

                // Add request headers.
                req3.Headers["api-version"] = "2011-11-01";
                req3.ContentType = "application/xml; charset=utf-8";
                req3.Method = "POST";

                // Write the request body to the request stream.
                ASCIIEncoding encoding3 = new ASCIIEncoding();
                byte[] byte3 = encoding3.GetBytes("<ArrayOfProperty xmlns=\"http://schemas.microsoft.com/HPCS2008R2/common\"><Property><Name>Name</Name><Value>My Job</Value></Property></ArrayOfProperty>");

                Stream reqStream3 = req3.GetRequestStream();

                reqStream3.Write(byte3, 0, byte3.Length);

                reqStream3.Close();

                HttpWebResponse response3 = req3.GetResponse() as HttpWebResponse;

                HttpStatusCode statusCode3 = response3.StatusCode;
                if (statusCode3 != HttpStatusCode.OK)
                {
                    HandleHttpErrors(response3);
                } 
                
                Console.WriteLine("Job {0} submitted.", jobId);

            }

            catch (Exception ex3)
            {
                Console.WriteLine("Exception: " + ex3.ToString());

                HandleException(ex3);
                throw;
            }

            do
            {

                // Pause for some time to allow job to run.
                Thread.Sleep(2000);

                try
                {

                    HttpWebRequest req4 = HttpWebRequest.Create("https://myheadnode.contoso.com/WindowsHpc/myheadnode/Job/" + jobId + "?Properties=State") as HttpWebRequest;

                    req4.Credentials = credsBasic;

                   // Add request headers.
                    req4.Headers["api-version"] = "2011-11-01";
                    req4.Method = "GET";

                    HttpWebResponse response4 = req4.GetResponse() as HttpWebResponse;

                    HttpStatusCode statusCode4 = response4.StatusCode;
                    if (statusCode4 != HttpStatusCode.OK)
                    {
                        HandleHttpErrors(response4);
                    }

                    // Get the job state from the response body.
                    Stream respStream4 = response4.GetResponseStream();

                    using (StreamReader reader4 = new StreamReader(respStream4))
                    {
                        string retString4 = reader4.ReadToEnd();

                        jobState = retString4.Substring(retString4.IndexOf("<Value>") + 7, retString4.IndexOf("</Value>") - retString4.IndexOf("<Value>") - 7);

                        reader4.Close();
                        respStream4.Close();

                    }

                    response4.Close();

                    // If the job failed or was canceled, exit without continuing with the 
                    // the final operation to get the job output.
                    if (jobState == "Failed" || jobState == "Canceled")
                    {
                        Console.WriteLine("Error: Job {0} completed with a state of {1}.", jobId, jobState);
                        Environment.Exit(-1);
                    }

                }

                catch (Exception ex4)
                {
                    Console.WriteLine("Exception: " + ex4.ToString());

                    HandleException(ex4);
                    throw;
                }

            // Exit the loop if the job is finished.
            } while (jobState != "Finished" && jobState != "Failed" && jobState != "Canceled");

            // Get the output of the task using the Get Task operation.
            try
            {

                HttpWebRequest req5 = HttpWebRequest.Create("https:// https://myheadnode.contoso.com/WindowsHpc/myheadnode/Job/" + jobId + "/Task/" + taskId + "?Properties=Output") as HttpWebRequest;

                req5.Credentials = credsBasic;

                // Add request headers.
                req5.Headers["api-version"] = "2011-11-01";
                req5.Method = "GET";

                HttpWebResponse response5 = req5.GetResponse() as HttpWebResponse;

                HttpStatusCode statusCode5 = response5.StatusCode;
                if (statusCode5 != HttpStatusCode.OK)
                {
                    HandleHttpErrors(response5);
                }

                // Get the task output from the response body.
                Stream respStream5 = response5.GetResponseStream();

                using (StreamReader reader5 = new StreamReader(respStream5))
                {
                    string retString5 = reader5.ReadToEnd();

                    string outputString = retString5.Substring(retString5.IndexOf("<Value>") + 7, retString5.IndexOf("</Value>") - retString5.IndexOf("<Value>") - 7);
                    Console.WriteLine("The output of task {0} in job {1} is: {2}", jobId, taskId, outputString);

                    reader5.Close();
                    respStream5.Close();

                }

                response5.Close();
                
            }

            catch (Exception ex5)
            {
                Console.WriteLine("Exception: " + ex5.ToString());

                HandleException(ex5);
                throw;
            }

        }

        [DataContract(Name = "HpcWebServiceFault", Namespace = "http://schemas.microsoft.com/HPCS2008R2/common")]
        [Serializable]
        public class HpcWebServiceFault
        {
            public HpcWebServiceFault()
            {
            }

            public HpcWebServiceFault(int code, string message, params KeyValuePair<string, string>[] values)
            {
                Code = code;
                Message = message;
                Values = values;
            }

            /// <summary>
            /// Gets the error code.
            /// </summary>
            [DataMember]
            public int Code
            {
                get;
                set;
            }

            /// <summary>
            /// Gets the reason for the error.
            /// </summary>
            [DataMember]
            public string Message
            {
                get;
                set;
            }

            /// <summary>
            /// Gets the context of the error.
            /// </summary>
            [DataMember]
            public KeyValuePair<string, string>[] Values
            {
                get;
                set;
            }
        }

        static void HandleException(Exception ex)
        {
            string message = ex.ToString();

            // First, try to parse the error string for the 
            // Windows HPC Server 2008 R2 with SP2 error format.
            if (ex is WebException)
            {
                WebException wex = ex as WebException;

                using (Stream responseStream = new MemoryStream())
                {
                    try
                    {
                        DataContractSerializer dcsString = new DataContractSerializer(typeof(string));

                        string callChain = wex.Response.Headers["x-ms-hpc-authoritychain"];

                        Console.WriteLine("Call chain for Windows Azure: " + callChain);

                        // Make a copy of the response so you can try the newer format for 
                        // the error response body later.
                        CopyStream(wex.Response.GetResponseStream(), responseStream);  

                        responseStream.Position = 0;

                        string errorBody = dcsString.ReadObject(responseStream) as string;

                        message = "Error from Windows HPC Server 2008 R2 with SP2: " + errorBody;
                    }
                    catch (Exception)
                    {
                        // If an exception occurred, try to parse the error string for the 
                        // Windows HPC Server 2008 R2 with SP3 error format.
                        if (responseStream.Length > 0)
                        {
                            try
                            {
                                DataContractSerializer dcsSP3 = new DataContractSerializer(typeof(HpcWebServiceFault));

                                responseStream.Position = 0;

                                object SP3_ErrorBodyObj = dcsSP3.ReadObject(responseStream);
                                HpcWebServiceFault fault = SP3_ErrorBodyObj as HpcWebServiceFault;

                                message = " Error from Windows HPC Server 2008 R2 with SP3. Code: " + fault.Code + "; Message: " + fault.Message;
                            }
                            catch (Exception)
                            {
                            }
                        }
                    }
                }

                Console.WriteLine("Exception received:");
                Console.WriteLine(message);
            }

            Console.WriteLine("Exception caught: " + ex.ToString());
            Console.WriteLine("Press Enter to exit.");
            Console.ReadLine();

            Environment.Exit(-1);
        }

        /// <summary>
        /// This callback function turns off the enforcement of X.509 certificate trust 
        /// chains, and enables the use of self-signed certificates.
        /// 
        /// If you want to enforce trust chains, do not register this callback function.
        /// </summary>
        /// <param name="obj"></param>
        /// <param name="cert"></param>
        /// <param name="chain"></param>
        /// <param name="err"></param>
        /// <returns></returns>
        public static bool DisableTrustChainEnforecment(object obj, X509Certificate cert, X509Chain chain, SslPolicyErrors err)
        {
            return true;
        }

        public static void HandleHttpErrors(HttpWebResponse response_err)
        {
            HttpStatusCode statusCode_err = response_err.StatusCode; 
            Stream respStream_err = response_err.GetResponseStream();

                using (StreamReader reader_err = new StreamReader(respStream_err))
                {
                    string message_err = reader_err.ReadToEnd();

                    reader_err.Close();
                    respStream_err.Close();

                    Console.WriteLine("Error. Code: {0}, Message: {1}", statusCode_err.ToString(), message_err);
                    Console.WriteLine("Press Enter to exit.");

                    Environment.Exit(-1);

                }

        }
        
        private static void CopyStream(Stream src, Stream dest)
        {
            byte[] buff = new byte[1024];
            int bytesRead;

            while ((bytesRead = src.Read(buff, 0, buff.Length)) > 0)
            {
                dest.Write(buff, 0, bytesRead);
            }
        }
    }

}

---

See Also

• Working with the Web Service Interface
• HPC Web Service API Reference
• Requesting Data

---