New-AzureHDInsightPigJobDefinition
Defines a new Pig job for an HDInsight service.
Note
The cmdlets referenced in this documentation are for managing legacy Azure resources that use Service Management APIs. See the Az PowerShell module for cmdlets to manage Azure Resource Manager resources.
Syntax
New-AzureHDInsightPigJobDefinition
[-Arguments <String[]>]
[-File <String>]
[-Files <String[]>]
[-Query <String>]
[-StatusFolder <String>]
[-Profile <AzureSMProfile>]
[<CommonParameters>] Description
This version of Azure PowerShell HDInsight is deprecated. These cmdlets will be removed by January 1, 2017. Please use the newer version of Azure PowerShell HDInsight.
For information about how to use the new HDInsight to create a cluster, see Create Linux-based clusters in HDInsight using Azure PowerShell (https://azure.microsoft.com/en-us/documentation/articles/hdinsight-hadoop-create-linux-clusters-azure-powershell/). For information about how to submit jobs by using Azure PowerShell and other approaches, see Submit Hadoop jobs in HDInsight (https://azure.microsoft.com/en-us/documentation/articles/hdinsight-submit-hadoop-jobs-programmatically/). For reference information about Azure PowerShell HDInsight, see Azure HDInsight Cmdlets.
The New-AzureHDInsightPigJobDefinition defines a Pig job for an Azure HDInsight service.
Examples
Example 1: Define a new Pig job
PS C:\>$0 = '$0';
PS C:\> $QueryString = "LOGS = LOAD 'wasb:///example/data/sample.log';" + "LEVELS = foreach LOGS generate REGEX_EXTRACT($0, '(TRACE|DEBUG|INFO|WARN|ERROR|FATAL)', 1) as LOGLEVEL;" + "FILTEREDLEVELS = FILTER LEVELS by LOGLEVEL is not null;" + "GROUPEDLEVELS = GROUP FILTEREDLEVELS by LOGLEVEL;" + "FREQUENCIES = foreach GROUPEDLEVELS generate group as LOGLEVEL, COUNT(FILTEREDLEVELS.LOGLEVEL) as COUNT;" + "RESULT = order FREQUENCIES by COUNT desc;" + "DUMP RESULT;"
PS C:\> $PigJobDefinition = New-AzureHDInsightPigJobDefinition -Query $QueryStringThe first command declares a string value, and then stores in the $0 variable.
The second command creates a Pig job query, and then stores it in the $QueryString variable.
The final command creates a Pig job definition that uses the query in $QueryString, and then stores the job definition in the $PigJobDefinition variable.
Parameters
-Arguments
Specifies an array of arguments for a Pig job. The arguments are passed as command-line arguments to each task.
| Type: | String[] |
| Aliases: | Args |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-File
Specifies the path to a file that contains a query to run. You can use this parameter instead of the Query parameter.
| Type: | String |
| Aliases: | QueryFile |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Files
Specifies a collection of files that are associated with a Pig job.
| Type: | String[] |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Profile
Specifies the Azure profile from which this cmdlet reads. If you do not specify a profile, this cmdlet reads from the local default profile.
| Type: | AzureSMProfile |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Query
Specifies a Pig job query.
| Type: | String |
| Aliases: | QueryText |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-StatusFolder
Specifies the location of the folder that contains standard outputs and error outputs for a job, including its exit code and task logs.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |