New-AzureHDInsightMapReduceJobDefinition

[This topic is pre-release documentation and is subject to change in future releases. Blank topics are included as placeholders.]

New-AzureHDInsightMapReduceJobDefinition

Defines a new Map Reduce job.

Syntax

Parameter Set: Default
New-AzureHDInsightMapReduceJobDefinition -ClassName <String> -JarFile <String> [-Arguments <String[]> ] [-Defines <Hashtable> ] [-Files <String[]> ] [-JobName <String> ] [-LibJars <String[]> ] [-StatusFolder <String> ] [ <CommonParameters>]

Detailed Description

Defines a new Map Reduce job to be run on an Azure HDInsight cluster.

This topic describes the cmdlet in the 0.8.1 version of the Microsoft Azure PowerShell module. To find out the version of the module you're using, from the Azure PowerShell console, type (get-module azure).version.

Parameters

-Arguments<String[]>

Arguments of the Hadoop job. The arguments will be passed as command line arguments to each task.

Aliases	Args
Required?	false
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

-ClassName<String>

Name of the job class in the job JAR file.

Aliases	Class
Required?	true
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

-Defines<Hashtable>

Hadoop configuration values to be set during the job execution.

Aliases	Params
Required?	false
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

-Files<String[]>

The collection of files required for the execution of the job. Use wasb file references here.

Aliases	none
Required?	false
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

-JarFile<String>

The fully qualified name of the JAR file that contains the code and dependencies of the MapReduce job.

Aliases	Jar
Required?	true
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

-JobName<String>

The name of the MapReduce job. Specifying the name is optional. If it is not explicitly specified the value of the ClassName parameter is used by default.

Aliases	Name
Required?	false
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

-LibJars<String[]>

LibJar references of the job.

Aliases	none
Required?	false
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

-StatusFolder<String>

Aliases	none
Required?	false
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

<CommonParameters>

This cmdlet supports the common parameters: -Verbose, -Debug, -ErrorAction, -ErrorVariable, -OutBuffer, and -OutVariable. For more information, see  about_CommonParameters (https://go.microsoft.com/fwlink/p/?LinkID=113216).

Inputs

The input type is the type of the objects that you can pipe to the cmdlet.

Outputs

The output type is the type of the objects that the cmdlet emits.

Examples

PS C:\> 

Define a Map Reduce job.

Define a new Map Reduce job to be run on a specified Azure HDInsight cluster..

PS C:\> $subid = (Get-AzureSubscription –Current).SubscriptionId$clustername = “yourcluster”$wordCountJob = New-AzureHDInsightMapReduceJobDefinition -JarFile "/example/apps/hadoop-examples.jar" -ClassName "wordcount" `   -Defines @{ "mapred.map.tasks"="3" } -Arguments "/example/data/gutenberg/davinci.txt", "/example/output/WordCount"$wordCountJob `    | Start-AzureHDInsightJob -Cluster $clustername `    | Wait-AzureHDInsightJob -Subscription $subid -WaitTimeoutInSeconds 3600 `    | Get-AzureHDInsightJobOutput -Cluster $clustername -Subscription $subid -StandardError