Edit

Share via

Get-AzureHDInsightJobOutput

Module:
Azure Module

Gets the log output for a job.

Note

The cmdlets referenced in this documentation are for managing legacy Azure resources that use Azure Service Manager (ASM) APIs. This legacy PowerShell module isn't recommended when creating new resources since ASM is scheduled for retirement. For more information, see Azure Service Manager retirement.

The Az PowerShell module is the recommended PowerShell module for managing Azure Resource Manager (ARM) resources with PowerShell.

Syntax

Default (Default) 

Get-AzureHDInsightJobOutput
    [-Certificate <X509Certificate2>]
    [-HostedService <String>]
    -Cluster <String>
    [-DownloadTaskLogs]
    [-Endpoint <Uri>]
    [-IgnoreSslErrors <Boolean>]
    -JobId <String>
    [-StandardError]
    [-StandardOutput]
    [-Subscription <String>]
    [-TaskLogsDirectory <String>]
    [-TaskSummary]
    [-Profile <AzureSMProfile>]
    [<CommonParameters>]

Description

This version of Azure PowerShell HDInsight is deprecated. These cmdlets will be removed by January 1, 2017. Please use the newer version of Azure PowerShell HDInsight.

For information about how to use the new HDInsight to create a cluster, see Create Linux-based clusters in HDInsight using Azure PowerShell. For information about how to submit jobs by using Azure PowerShell and other approaches, see Submit Hadoop jobs in HDInsight. For reference information about Azure PowerShell HDInsight, see Azure HDInsight Cmdlets.

The Get-AzureHDInsightJobOutput cmdlet gets the log output for a job from the storage account associated with a cluster. You can get various types of job logs including standard output, standard error, task logs, and a summary of the task logs.

Examples

Example 1: Get job output

PS C:\>$SubId = (Get-AzureSubscription -Current).SubscriptionId
PS C:\> $ClusterName = "MyCluster"
PS C:\> $WordCountJob = New-AzureHDInsightMapReduceJobDefinition -JarFile "/Example/Apps/Hadoop-examples.jar" -ClassName "Wordcount" -Defines @{ "mapred.map.tasks" = "3" } -Arguments "/Example/Data/Gutenberg/Davinci.txt", "/Example/Output/WordCount" $WordCountJob
    | Start-AzureHDInsightJob -Subscription $SubId -Cluster $ClusterName
    | Wait-AzureHDInsightJob -Subscription $SubId -WaitTimeoutInSeconds 3600
    | Get-AzureHDInsightJobOutput -Cluster $ClusterName -StandardError

The first command gets the ID of the current subscription, and then stores it in the $SubId variable.

The second command stores the name MyCluster in the $Clustername variable.

The third command creates a MapReduce job definition, and then stores it in the $WordCountJob variable. The command passes the job in $WordCountJob to the Start-AzureHDInsightJob cmdlet to start the job. It also passes $WordCountJob to the Wait-AzureHDInsightJob cmdlet to wait for the job to finish, and then it uses Get-AzureHDInsightJobOutput to get the job output.

Parameters

-Certificate

Specifies the management certificate for an Azure subscription.

Parameter properties

Type:X509Certificate2
Default value:None
Supports wildcards:False
DontShow:False
Aliases:Cert

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-Cluster

Specifies a cluster. This cmdlet gets job logs from the cluster that this parameter specifies.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False
Aliases:ClusterName

Parameter sets

(All)
Position:Named
Mandatory:True
Value from pipeline:True
Value from pipeline by property name:True
Value from remaining arguments:False

-DownloadTaskLogs

Indicates that this cmdlet gets the task logs for a job.

Parameter properties

Type:SwitchParameter
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-Endpoint

Specifies the endpoint to use to connect to Azure. If you do not specify this parameter, this cmdlet uses the default endpoint.

Parameter properties

Type:Uri
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-HostedService

Specifies the namespace of an HDInsight service if you do not want to use the default namespace.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False
Aliases:CloudServiceName

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-IgnoreSslErrors

Indicates whether Secure Sockets Layer (SSL) errors are ignored.

Parameter properties

Type:Boolean
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-JobId

Specifies the ID of the job to get.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False
Aliases:Id

Parameter sets

(All)
Position:Named
Mandatory:True
Value from pipeline:True
Value from pipeline by property name:True
Value from remaining arguments:False

-Profile

Specifies the Azure profile from which this cmdlet reads. If you do not specify a profile, this cmdlet reads from the local default profile.

Parameter properties

Type:AzureSMProfile
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-StandardError

Indicates that this cmdlet gets the StdErr output of a job.

Parameter properties

Type:SwitchParameter
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-StandardOutput

Indicates that this cmdlet gets the SdtOut output of a job.

Parameter properties

Type:SwitchParameter
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-Subscription

Specifies the subscription that contains the HDInsight cluster to get.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False
Aliases:Sub

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-TaskLogsDirectory

Specifies a local folder in which to store tasks logs.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False
Aliases:LogsDir

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-TaskSummary

Indicates that this cmdlets gets the task log summary.

Parameter properties

Type:SwitchParameter
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

CommonParameters

This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable, -InformationAction, -InformationVariable, -OutBuffer, -OutVariable, -PipelineVariable, -ProgressAction, -Verbose, -WarningAction, and -WarningVariable. For more information, see about_CommonParameters.