Add-AzureHDInsightConfigValues

[This topic is pre-release documentation and is subject to change in future releases. Blank topics are included as placeholders.]

Add-AzureHDInsightConfigValues

Adds a Hadoop configuration value customization and/or a Hive shared library customization to the Windows Azure HDInsight cluster configuration.

Syntax

Parameter Set: Set Default Storage Account
Add-AzureHDInsightConfigValues [-Config] <AzureHDInsightConfig> [-Core <Hashtable> ] [-Hdfs <Hashtable> ] [-Hive <AzureHDInsightHiveConfiguration> ] [-MapReduce <AzureHDInsightMapReduceConfiguration> ] [-Oozie <AzureHDInsightOozieConfiguration> ] [-Yarn <Hashtable> ] [ <CommonParameters>]

Detailed Description

Adds a Hadoop configuration value customization, such as core-site.xml or hive-site.xml, and/or a Hive shared library customization to the Windows Azure HDInsight cluster configuration.

This topic describes the cmdlet in the 0.8.1 version of the Microsoft Azure PowerShell module. To find out the version of the module you're using, from the Azure PowerShell console, type (get-module azure).version.

Parameters

-Config<AzureHDInsightConfig>

The configuration object to which Hadoop configuration is added.

Aliases	none
Required?	true
Position?	1
Default Value	none
Accept Pipeline Input?	True (ByValue)
Accept Wildcard Characters?	false

-Core<Hashtable>

Set of customized Hadoop configuration values for core-site.xml.

Aliases	none
Required?	false
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

-Hdfs<Hashtable>

Set of customized Hadoop configuration values for hdfs-site.xml.

Aliases	none
Required?	false
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

-Hive<AzureHDInsightHiveConfiguration>

Customization object for Hadoop Hive service, including a set of Hadoop configuration values for hive-site.xml and Hive shared libraries.

Aliases	none
Required?	false
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

-MapReduce<AzureHDInsightMapReduceConfiguration>

Customization object for map reduce and the capacity scheduler.

Aliases	none
Required?	false
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

-Oozie<AzureHDInsightOozieConfiguration>

Customization object for Hadoop Oozie service. Includes set of Hadoop configuration values for oozie-site.xml.

Aliases	none
Required?	false
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

-Yarn<Hashtable>

Customization object for Hadoop YARN service.

Aliases	none
Required?	false
Position?	named
Default Value	none
Accept Pipeline Input?	false
Accept Wildcard Characters?	false

<CommonParameters>

This cmdlet supports the common parameters: -Verbose, -Debug, -ErrorAction, -ErrorVariable, -OutBuffer, and -OutVariable. For more information, see  about_CommonParameters (https://go.microsoft.com/fwlink/p/?LinkID=113216).

Inputs

The input type is the type of the objects that you can pipe to the cmdlet.

Outputs

The output type is the type of the objects that the cmdlet emits.

Examples

Cluster configuration

Set Hive,Oozie, and MapReduce configuration values for an HDInsight cluster.

PS C:\> $hiveConfigvalues = new-object 'Microsoft.WindowsAzure.Management.HDInsight.Cmdlet.DataObjects.AzureHDInsightHiveConfiguration'$hiveConfigvalues.Configuration = @{ “hive.exec.compress.output”=”true” }$hiveConfigvalues.AdditionalLibraries = new-object 'Microsoft.WindowsAzure.Management.HDInsight.Cmdlet.DataObjects.AzureHDInsightDefaultStorageAccount'$hiveConfigvalues.AdditionalLibraries.StorageAccountName = "yourstorageaccount.blob.core.windows.net" $hiveConfigvalues.AdditionalLibraries.StorageAccountKey = (Get-AzureStorageKey yourstorageaccount).Primary$hiveConfigvalues.AdditionalLibraries.StorageContainerName = "yoursharedlibcontainer"$oozieConfigvalues = new-object 'Microsoft.WindowsAzure.Management.HDInsight.Cmdlet.DataObjects.AzureHDInsightOozieConfiguration'$oozieConfigvalues.Configuration = @{ “hive.exec.compress.output”=”true” }$mapredConfigvalues = new-object 'Microsoft.WindowsAzure.Management.HDInsight.Cmdlet.DataObjects.AzureHDInsightMapReduceConfiguration'$mapredConfigvalues.Configuration = @{ “mapred.map.max.attempts”=”2” }$mapredConfigvalues.CapacitySchedulerConfiguration = @{ “mapred.capacity-scheduler.init-poll-interval”=”1000” }$config = New-AzureHDInsightClusterConfig -ClusterSizeInNodes 4 `    | Set-AzureHDInsightDefaultStorage -StorageAccountName yourstorageaccount.blob.core.windows.net `        -StorageAccountKey (Get-AzureStorageKey yourstorageaccount).Primary -StorageContainerName " yourstoragecontainer" `    | Add-AzureHDInsightConfigValues -Core @{ “io.file.buffer.size”=”300000” } -MapReduce $mapredConfigvalues -Hive $hiveConfigvalues -Oozie $oozieConfigvalues$config | New-AzureHDInsightCluster -Credential $creds -Name yourcluster -Location "North Europe"