Cluster REST API in Azure HDInsight
Creates a cluster in the specified subscription.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
PUT | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}?api-version={api-version} |
The following example shows the request body for creating a Linux-based hadoop cluster. For examples of creating clusters in other ways, see the Examples section below.
{
"id":"/subscriptions/{subscription-id}/resourceGroups/myresourcegroup1/providers/Microsoft.HDInsight/clusters/mycluster",
"name":"mycluster",
"type":"Microsoft.HDInsight/clusters",
"location": "location-name",
"tags": { "tag1": "value1", "tag2": "value2" },
"properties": {
"clusterVersion": "3.2",
"osType": "Linux",
"clusterDefinition": {
"kind": "hadoop",
"configurations": {
"gateway": {
"restAuthCredential.isEnabled": true,
"restAuthCredential.username": "http-user",
"restAuthCredential.password": "password"
},
"core-site": {
"fs.defaultFS": "wasb://container@storageaccount.blob.core.windows.net",
"fs.azure.account.key.storageaccount.blob.core.windows.net": storage-account-key"
}
}
},
"computeProfile": {
"roles": [
{
"name": "headnode",
"targetInstanceCount": 2,
"hardwareProfile": {
"vmSize": "Large"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "username",
"sshProfile": {
"publicKeys": [
{ "certificateData": "ssh-rsa key" }
]
}
}
}
},
{
"name": "workernode",
"targetInstanceCount": 1,
"hardwareProfile": {
"vmSize": "Large"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "username",
"sshProfile": {
"publicKeys": [
{ "certificateData": " ssh-rsa key" }
]
}
}
}
},
{
"name": "zookeepernode",
"targetInstanceCount": 3,
"hardwareProfile": {
"vmSize": "Small"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "username",
"sshProfile": {
"publicKeys": [
{ "certificateData": "ssh-rsa key" }
]
}
}
}
}
]
}
}
}
Element name | Required | Type | Description |
---|---|---|---|
id | Yes | String | Specifies the resource identifier of the cluster. |
name | Yes | String | Specifies the name of the cluster. |
type | Yes | String | Specifies the type of the cluster. |
location | Yes | String | Specifies the supported Azure location where the cluster should be created. For more information, see List all of the available geo-locations. |
tags | No | String | Specifies the tags that will be assigned to the cluster. For more information about using tags, see Using tags to organize your Azure resources. |
Properties | Yes | Complex Type | Specifies the properties of the cluster. |
Element name | Required | Type | Description |
---|---|---|---|
clusterVersion | Yes | String | Specifies the cluster version |
osType | Yes | String | Specifies the Operating system for the cluster. Valid values are Linux and Windows |
clusterDefinition | Yes | Complex Type | Specifies information about the cluster type and configurations |
computeProfile | Yes | Complex Type | Specifies information about the cluster topology and associated role properties |
Element name | Required | Type | Description |
---|---|---|---|
kind | Yes | String | Specifies the cluster type. Valid values are hadoop, hbase, storm & spark |
configurations | Yes | Dictionary | This element is a dictionary of configuration type and its associated value dictionary. gateway configuration type is used to configure the http user used for connecting to web APIs and the Ambari portal core-site configuration type is used to configure the default storage account for the cluster |
Element name | Required | Type | Description |
---|---|---|---|
clusterVersion | Yes | String | Specifies the cluster version |
role | Yes | Array of Complex Type (role) | Specifies information about roles in the cluster |
Element name | Required | Type | Description |
---|---|---|---|
name | Yes | String | Specifies the role name |
targetInstanceCount | Yes | Integer | Specifies the target instance count for the role |
hardwareProfile | Yes | Complex Type | Specifies information about the hardware profile for the role |
osProfile | Yes | Complex Type | Specifies information about the os profile for the role |
Element name | Required | Type | Description |
---|---|---|---|
vmSize | Yes | String | Specifies the size of the VM. Refer to HDInsight configuration options (once on this link, scroll down to Node pricing tiers) for valid sizes |
Element name | Required | Type | Description |
---|---|---|---|
linuxOperatingSystemProfile | No | Complex Type | Specifies the linux OS-related settings |
windowsOperatingSystemProfile | No | Complex Type | Specifies windows OS-related settings |
virtualNetworkProfile | No | Complex Type | Specifies virtual network-related settings if the cluster is being deployed in a virtual network in the user’s subscription |
scriptActions | No | Array of Complex Type | List of script actions to execute on the cluster |
Element name | Required | Type | Description |
---|---|---|---|
Username | Yes | String | SSH user name |
sshProfile | No | Complex Type | Specifies the SSH key. One of sshProfile or Password is required. |
Password | No | String | Specifies the SSH password One of sshProfile or Password is required. |
Element name | Required | Type | Description |
---|---|---|---|
publicKeys | Yes | Array | Contains a list of certificateData objects. The value is an ssh-rsa public key |
Element name | Required | Type | Description |
---|---|---|---|
rdpSettings | No | Complex Type | Specifies RDP settings for windows clusters |
Element name | Required | Type | Description |
---|---|---|---|
username | Yes | String | Specifies the RDP user name |
password | Yes | String | Specifies the password for the RDP user |
expiryDate | Yes | Date | Expiry date for the RDP credentials |
Element name | Required | Type | Description |
---|---|---|---|
id | Yes | String | Virtual Network Resource Id |
subnet | Yes | String | Specifies the subnet name |
Element name | Required | Type | Description |
---|---|---|---|
name | Yes | String | Friendly name for the script action |
uri | Yes | String | URL to the script action file |
parameters | No | String | Arguments to pass when executing the script action file |
If validation is complete and the request is accepted, the operation will return 200 (OK).
Status code: 200 OK
Response body for a linux cluster creates using ssh key:
{
"id":"/subscriptions/{subscription-id}/resourceGroups/myresourcegroup1/providers/Microsoft.HDInsight/clusters/mycluster",
"name":"mycluster",
"type":"Microsoft.HDInsight/clusters",
"location": "location-name",
"tags": { "tag1": "value1", "tag2": "value2" },
"properties": {
"clusterVersion": "3.2",
"osType": "Linux",
"provisioningState": "InProgress",
"clusterState": "Accepted",
"createdDate": "2015-09-23",
"quotaInfo": {
"coresUsed": 20
}
"clusterDefinition": {
"kind": "hadoop"
},
"computeProfile": {
"roles": [
{
"name": "headnode",
"targetInstanceCount": 2,
"hardwareProfile": {
"vmSize": "Large"
}
},
{
"name": "workernode",
"targetInstanceCount": 1,
"hardwareProfile": {
"vmSize": "Large"
}
},
{
"name": "zookeepernode",
"targetInstanceCount": 3,
"hardwareProfile": {
"vmSize": "Small"
}
}
]
}
}
}
Element name | Type | Description |
---|---|---|
provisioningState | String | Indicates the current provisioning state. |
clusterState | String | Indicates the more detailed HDInsight cluster state while provisioning is in progress. |
createdDate | Date | Datetime when the cluster create request was received |
quotaInfo | Complex Type | Specifies the coresUsed by the cluster |
errors | Array of error messages | Contains the error message if provisioningState = ‘failed" |
connectivityEndpoints | Complex Type | Specifies the public endpoints for the cluster |
Element name | Type | Description |
---|---|---|
name | String | Friendly name for the connectivity endpoint |
protocol | String | Specifies the Protocol to use (example: HTTPS, SSH) |
location | String | Specifies the URL to connect |
port | int | Specifies the port to connect |
Create a premium domain-joined cluster with Apache Ranger. User needs to provide SecurityProfile in the request body to create a secure cluster.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
PUT | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}?api-version={api-version} |
The following example shows the request body for creating a Linux based, premium, domain-joined Hadoop cluster.
{
"id": "/subscriptions/{ subscription-id }/resourceGroups/myresourcegroup1/providers/Microsoft.HDInsight/ clusters/mycluster ", "
name "
: "mycluster",
"type": "Microsoft.HDInsight/clusters",
"location": "location-name",
"tags": {
"tag1": "value1",
"tag2": "value2"
},
"properties": {
"clusterVersion": "3.5",
"osType": "Linux",
"tier": "premium",
"clusterDefinition": {
"kind": "hadoop",
"configurations": {
"gateway": {
"restAuthCredential.isEnabled": true,
"restAuthCredential.username": "http-user",
"restAuthCredential.password": "password"
},
"core-site": {
"fs.defaultFS": "wasb://container@storageaccount.blob.core.windows.net",
"fs.azure.account.key.storageaccount.blob.core.windows.net": "storage-account-key"
}
}
},
"securityProfile": {
"directoryType": "ActiveDirectory",
"domain": "mydomain.com",
"organizationalUnitDN": "OU=Hadoop,DC=mydomain,DC=COM",
"ldapsUrls": ["ldaps://mydomain.com:636"],
"domainUsername": "clusteradmin@mydomain.com",
"domainUserPassword": "password",
"clusterUsersGroupDNs": ["ADGroup1", "ADGroup2"]
},
"computeProfile": {
"roles": [
{
"name": "headnode",
"targetInstanceCount": 2,
"hardwareProfile": {
"vmSize": "Large"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "username",
"sshProfile": {
"publicKeys": [
{
"certificateData": "ssh-rsa key"
}
]
}
}
},
"virtualNetworkProfile": {
"id": "/subscriptions/mysubscriptionid/resourceGroups/myrresourcegroup/providers/Microsoft.Network/virtualNetworks/myvirtualnetwork",
"subnet": "/subscriptions/mysubscriptionid /resourceGroups/myresourcegroup/providers/Microsoft.Network/virtualNetworks/myvirtualnetwork/subnets/mysubnet"
}
},
{
"name": "workernode",
"targetInstanceCount": 1,
"hardwareProfile": {
"vmSize": "Large"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "username",
"sshProfile": {
"publicKeys": [
{
"certificateData": " ssh-rsa key"
}
]
}
}
},
"virtualNetworkProfile": {
"id": "/subscriptions/mysubscriptionid/resourceGroups/myrresourcegroup/providers/Microsoft.Network/virtualNetworks/myvirtualnetwork",
"subnet": "/subscriptions/mysubscriptionid /resourceGroups/myresourcegroup/providers/Microsoft.Network/virtualNetworks/myvirtualnetwork/subnets/mysubnet"
}
},
{
"name": "zookeepernode",
"targetInstanceCount": 3,
"hardwareProfile": {
"vmSize": "Small"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "username",
"sshProfile": {
"publicKeys": [
{
"certificateData": "ssh-rsa key"
}
]
}
},
"virtualNetworkProfile": {
"id": "/subscriptions/mysubscriptionid/resourceGroups/myrresourcegroup/providers/Microsoft.Network/virtualNetworks/myvirtualnetwork",
"subnet": "/subscriptions/mysubscriptionid /resourceGroups/myresourcegroup/providers/Microsoft.Network/virtualNetworks/myvirtualnetwork/subnets/mysubnet"
}
}
}
]
}
}
}
Element name | Required | Type | Description |
---|---|---|---|
id | Yes | String | Specifies the resource identifier of the cluster. |
name | Yes | String | Specifies the name of the cluster. |
type | Yes | String | Specifies the type of the cluster. |
location | Yes | String | Specifies the supported Azure location where the cluster should be created. For more information, see List all of the available geo-locations. |
tags | No | String | Specifies the tags that will be assigned to the cluster. For more information about using tags, see Using tags to organize your Azure resources. |
Properties | Yes | Complex Type | Specifies the properties of the cluster. |
Element name | Required | Type | Description |
---|---|---|---|
clusterVersion | Yes | String | Specifies the cluster version |
osType | Yes | String | Specifies the Operating system for the cluster. Valid value is Linux because only Linux cluster types can join an Azure AD domain. |
tier | No | String | Default value is standard. Valid values are standard and premium. If no value is specified, the value is assumed to be standard. Specifies the Tier for the cluster. Domain joined clusters are only supported in premium tier |
clusterDefinition | Yes | Complex Type | Specifies information about the cluster type and configurations |
computeProfile | Yes | Complex Type | Specifies information about the cluster topology and associated role properties |
securityProfile | No | Complex Type | If a secure, domain-joined cluster is being created, this element specifies the Active Directory related settings |
Element name | Required | Type | Description |
---|---|---|---|
kind | Yes | String | Specifies the cluster type. Valid values are hadoop, hbase, storm & spark |
configurations | Yes | Dictionary | This element is a dictionary of configuration type and its associated value dictionary. gateway configuration type is used to configure the http user used for connecting to web APIs and the Ambari portal core-site configuration type is used to configure the default storage account for the cluster |
Element name | Required | Type | Description |
---|---|---|---|
clusterVersion | Yes | String | Specifies the cluster version |
role | Yes | Array of Complex Type (role) | Specifies information about roles in the cluster |
Element name | Required | Type | Description |
---|---|---|---|
directoryType | Yes | String | Type of LDAP directory that is used. Currently "ActiveDirectory" is the only supported value. |
domain | Yes | String | Active Directory domain for the cluster |
organizationalUnitDN | Yes | String | Distinguished name of the organizational unit in the Active directory where user and computer accounts will be created |
ldapsUrls | Yes | Array of String | URLs of one or multiple LDAPS servers for the Active Directory |
domainUserName | Yes | String | A domain user account with sufficient permissions for creating the cluster. It should be in user@domain format |
domainUserPassword | Yes | String | Password for the domain user account |
clusterUsersGroupDNS | No | Array of String | Distinguished names of the Active Directory groups that will be available in Ambari and Apache Ranger |
Element name | Required | Type | Description |
---|---|---|---|
name | Yes | String | Specifies the role name |
targetInstanceCount | Yes | Integer | Specifies the target instance count for the role |
hardwareProfile | Yes | Complex Type | Specifies information about the hardware profile for the role |
osProfile | Yes | Complex Type | Specifies information about the os profile for the role |
Element name | Required | Type | Description |
---|---|---|---|
vmSize | Yes | String | Specifies the size of the VM. Refer to HDInsight configuration options (once on this link, scroll down to Node pricing tiers) for valid sizes |
Element name | Required | Type | Description |
---|---|---|---|
linuxOperatingSystemProfile | No | Complex Type | Specifies the linux OS-related settings |
virtualNetworkProfile | No | Complex Type | Specifies virtual network-related settings if the cluster is being deployed in a virtual network in the user’s subscription |
scriptActions | No | Array of Complex Type | List of script actions to execute on the cluster |
Element name | Required | Type | Description |
---|---|---|---|
Username | Yes | String | SSH user name |
sshProfile | No | Complex Type | Specifies the SSH key. One of sshProfile or Password is required. |
Password | No | String | Specifies the SSH password One of sshProfile or Password is required. |
Element name | Required | Type | Description |
---|---|---|---|
publicKeys | Yes | Array | Contains a list of certificateData objects. The value is an ssh-rsa public key |
Element name | Required | Type | Description |
---|---|---|---|
id | Yes | String | Virtual Network Resource Id |
subnet | Yes | String | Specifies the subnet name |
Element name | Required | Type | Description |
---|---|---|---|
name | Yes | String | Friendly name for the script action |
uri | Yes | String | URL to the script action file |
parameters | No | String | Arguments to pass when executing the script action file |
If validation is complete and the request is accepted, the operation will return 200 (OK).
Status code: 200 OK
Response body for a linux cluster creates using ssh key:
{
"id":"/subscriptions/{subscription-id}/resourceGroups/myresourcegroup1/providers/Microsoft.HDInsight/clusters/mycluster",
"name":"mycluster",
"type":"Microsoft.HDInsight/clusters",
"location": "location-name",
"tags": { "tag1": "value1", "tag2": "value2" },
"properties": {
"clusterVersion": "3.2",
"osType": "Linux",
"provisioningState": "InProgress",
"clusterState": "Accepted",
"createdDate": "2015-09-23",
"quotaInfo": {
"coresUsed": 20
}
"clusterDefinition": {
"kind": "hadoop"
},
"computeProfile": {
"roles": [
{
"name": "headnode",
"targetInstanceCount": 2,
"hardwareProfile": {
"vmSize": "Large"
}
},
{
"name": "workernode",
"targetInstanceCount": 1,
"hardwareProfile": {
"vmSize": "Large"
}
},
{
"name": "zookeepernode",
"targetInstanceCount": 3,
"hardwareProfile": {
"vmSize": "Small"
}
}
]
}
}
}
Element name | Type | Description |
---|---|---|
provisioningState | String | Indicates the current provisioning state. |
clusterState | String | Indicates the more detailed HDInsight cluster state while provisioning is in progress. |
createdDate | Date | Datetime when the cluster create request was received |
quotaInfo | Complex Type | Specifies the coresUsed by the cluster |
errors | Array of error messages | Contains the error message if provisioningState = ‘failed" |
connectivityEndpoints | Complex Type | Specifies the public endpoints for the cluster |
Element name | Type | Description |
---|---|---|
name | String | Friendly name for the connectivity endpoint |
protocol | String | Specifies the Protocol to use (example: HTTPS, SSH) |
location | String | Specifies the URL to connect |
port | int | Specifies the port to connect |
Creates a cluster in the specified subscription with Azure Data Lake Store as the default filesystem. Provide a ClusterIdentity object in the request body and configure the default-filesystem property with appropriate Data Lake Store URL.
Azure Data Lake can be configured as the default filesystem for cluster versions starting from 3.5 inclusive.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
PUT | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}?api-version={api-version}} |
The following example shows the request body for creating a Linux-based Hadoop cluster using Azure Data Lake Store as the default filesystem for the cluster.
{
"id": "/subscriptions/{subscription-id}/resourceGroups/myresourcegroup1/providers/Microsoft.HDInsight/clusters/mycluster",
"name": "mycluster",
"type": "Microsoft.HDInsight/clusters",
"location": "location-name",
"tags": {
"tag1": "value1",
"tag2": "value2"
},
"properties": {
"clusterVersion": "3.5",
"osType": "Linux",
"tier": "Standard",
"clusterDefinition": {
"kind": "hadoop",
"configurations": {
"gateway": {
"restAuthCredential.isEnabled": true,
"restAuthCredential.username": "http-user",
"restAuthCredential.password": "password"
},
"clusterIdentity": {
"clusterIdentity.applicationId": "applicationId",
"clusterIdentity.certificate": "certificate-contents-in-base64-encoding",
"clusterIdentity.aadTenantId": "aad-tenant-id",
"clusterIdentity.resourceUri": "https://KonaCompute.net/",
"clusterIdentity.certificatePassword": "certificate-password"
},
"core-site": {
"fs.defaultFS": "adl://home",
"dfs.adls.home.hostname": "yourstorageaccount.azuredatalakestore.net",
"dfs.adls.home.mountpoint": "/path/to/cluster/root"
}
}
},
"computeProfile": {
"roles": [
{
"name": "headnode",
"targetInstanceCount": 2,
"hardwareProfile": {
"vmSize": "Large"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "username",
"sshProfile": {
"publicKeys": [
{
"certificateData": "ssh-rsa key"
}
]
}
}
}
},
{
"name": "workernode",
"targetInstanceCount": 1,
"hardwareProfile": {
"vmSize": "Large"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "username",
"sshProfile": {
"publicKeys": [
{
"certificateData": " ssh-rsa key"
}
]
}
}
}
},
{
"name": "zookeepernode",
"targetInstanceCount": 3,
"hardwareProfile": {
"vmSize": "Small"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "username",
"sshProfile": {
"publicKeys": [
{
"certificateData": "ssh-rsa key"
}
]
}
}
}
}
]
}
}
}
Element name | Required | Type | Description |
---|---|---|---|
id | Yes | String | Specifies the resource identifier of the cluster. |
name | Yes | String | Specifies the name of the cluster. |
type | Yes | String | Specifies the type of the cluster. |
location | Yes | String | Specifies the supported Azure location where the cluster should be created. For more information, see List all of the available geo-locations. |
tags | No | String | Specifies the tags that will be assigned to the cluster. For more information about using tags, see Using tags to organize your Azure resources. |
Properties | Yes | Complex Type | Specifies the properties of the cluster. |
Element name | Required | Type | Description |
---|---|---|---|
clusterVersion | Yes | String | Specifies the cluster version |
osType | Yes | String | Specifies the Operating system for the cluster. Valid value is Linux |
tier | No | String | Specifies the tier for the cluster. Valid values are standard and premium. If tier is not specified, cluster will be of standard tier. |
clusterDefinition | Yes | Complex Type | Specifies information about the cluster type and configurations |
computeProfile | Yes | Complex Type | Specifies information about the cluster topology and associated role properties |
Element name | Required | Type | Description |
---|---|---|---|
kind | Yes | String | Specifies the cluster type. Valid values are hadoop, hbase, storm & spark |
configurations | Yes | Dictionary | This element is a dictionary of configuration type and its associated value dictionary. gateway configuration type is used to configure the http user used for connecting to web APIs and the Ambari portal core-site configuration type is used to configure the default storage account for the cluster |
Element name | Required | Type | Description |
---|---|---|---|
clusterVersion | Yes | String | Specifies the cluster version |
role | Yes | Array of Complex Type (role) | Specifies information about roles in the cluster |
Element name | Required | Type | Description |
---|---|---|---|
name | Yes | String | Specifies the role name |
targetInstanceCount | Yes | Integer | Specifies the target instance count for the role |
hardwareProfile | Yes | Complex Type | Specifies information about the hardware profile for the role |
osProfile | Yes | Complex Type | Specifies information about the os profile for the role |
Element name | Required | Type | Description |
---|---|---|---|
vmSize | Yes | String | Specifies the size of the VM. Refer to HDInsight configuration options (once on this link, scroll down to Node pricing tiers) for valid sizes |
Element name | Required | Type | Description |
---|---|---|---|
linuxOperatingSystemProfile | No | Complex Type | Specifies the linux OS-related settings |
virtualNetworkProfile | No | Complex Type | Specifies virtual network-related settings if the cluster is being deployed in a virtual network in the user’s subscription |
scriptActions | No | Array of Complex Type | List of script actions to execute on the cluster |
Element name | Required | Type | Description |
---|---|---|---|
Username | Yes | String | SSH user name |
sshProfile | No | Complex Type | Specifies the SSH key. One of sshProfile or Password is required. |
Password | No | String | Specifies the SSH password One of sshProfile or Password is required. |
Element name | Required | Type | Description |
---|---|---|---|
publicKeys | Yes | Array | Contains a list of certificateData objects. The value is an ssh-rsa public key |
Element name | Required | Type | Description |
---|---|---|---|
id | Yes | String | Virtual Network Resource Id |
subnet | Yes | String | Specifies the subnet name |
Element name | Required | Type | Description |
---|---|---|---|
name | Yes | String | Friendly name for the script action |
uri | Yes | String | URL to the script action file |
parameters | No | String | Arguments to pass when executing the script action file |
If validation is complete and the request is accepted, the operation will return 200 (OK).
Status code: 200 OK
Response body for a linux cluster creates using ssh key:
{
"id": "/subscriptions/{subscription ID}/resourceGroups/rg1/providers/Microsoft.HDInsightCurrent/clusters/mycluster",
"name": "mycluster",
"type": "Microsoft.HDInsightCurrent/clusters",
"location": "East Asia",
"etag": "{value}",
"tags": null,
"properties": {
"clusterVersion": "3.5.1000.0",
"osType": "Linux",
"clusterDefinition": {
"kind": "hadoop"
},
"computeProfile": {
"roles": [
{
"name": "headnode",
"targetInstanceCount": 2,
"hardwareProfile": {
"vmSize": "Standard_D3"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "myuser"
}
}
},
{
"name": "workernode",
"targetInstanceCount": 2,
"hardwareProfile": {
"vmSize": "Standard_D3"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "myuser"
}
}
}
]
},
"provisioningState": "InProgress",
"clusterState": "Accepted",
"createdDate": "2016-11-15T09:21:47.61",
"quotaInfo": {
"coresUsed": 16
},
"tier": "standard"
}
}
Element name | Type | Description |
---|---|---|
provisioningState | String | Indicates the current provisioning state. |
clusterState | String | Indicates the more detailed HDInsight cluster state while provisioning is in progress. |
createdDate | Date | Datetime when the cluster create request was received |
quotaInfo | Complex Type | Specifies the coresUsed by the cluster |
errors | Array of error messages | Contains the error message if provisioningState = ‘failed" |
connectivityEndpoints | Complex Type | Specifies the public endpoints for the cluster |
Element name | Type | Description |
---|---|---|
name | String | Friendly name for the connectivity endpoint |
protocol | String | Specifies the Protocol to use (example: HTTPS, SSH) |
location | String | Specifies the URL to connect |
port | int | Specifies the port to connect |
Deletes an HDInsight cluster.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
DELETE | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}?api-version={api-version} |
HTTP 202 (Accepted) to indicate that the operation will complete asynchronously. Async polling will return a 204 (NoContent) once the operation completes successfully.
To track progress of a delete cluster request, see Asynchronous Operations (202 Accepted and Location header)
Gets the details/properties of the specified cluster.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
GET | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}?api-version={api-version} |
The operation will return 200 (OK) if the request is completed successfully
Status code: 200 OK
Response body is the same as create cluster.
Response body for linux cluster details
{
"id":"/subscriptions/{subscription-id}/resourceGroups/myresourcegroup1/providers/Microsoft.HDInsight/clusters/mycluster",
"name":"mycluster",
"type":"Microsoft.HDInsight/clusters",
"location": "location-name",
"tags": { "tag1": "value1", "tag2": "value2" },
"properties": {
"clusterVersion": "3.2",
"osType": "Linux",
"provisioningState": "InProgress",
"clusterState": "Accepted",
"createdDate": "2015-09-23",
"quotaInfo": {
"coresUsed": 20
}
"clusterDefinition": {
"kind": "hadoop"
},
"computeProfile": {
"roles": [
{
"name": "headnode",
"targetInstanceCount": 2,
"hardwareProfile": {
"vmSize": "Large"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "sshuser"
}
}
},
{
"name": "workernode",
"targetInstanceCount": 1,
"hardwareProfile": {
"vmSize": "Large"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "sshuser"
}
}
},
{
"name": "zookeepernode",
"targetInstanceCount": 3,
"hardwareProfile": {
"vmSize": "Small"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "sshuser"
}
}
}
]
}
}
}
Element name | Type | Description |
---|---|---|
provisioningState | String | Indicates the current provisioning state. |
clusterState | String | Indicates the more detailed HDInsight cluster state while provisioning is in progress. |
createdDate | Date | Datetime when the cluster create request was received |
quotaInfo | Complex Type | Specifies the coresUsed by the cluster |
errors | Array of error messages | Contains the error message if provisioningState = ‘failed" |
connectivityEndpoints | Complex Type | Specifies the public endpoints for the cluster |
Element name | Type | Description |
---|---|---|
name | String | Friendly name for the connectivity endpoint |
protocol | String | Specifies the Protocol to use (example: HTTPS, SSH) |
location | String | Specifies the URL to connect |
port | int | Specifies the port to connect |
Retrieves all cluster configuration details (including sensitive information).
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
POST | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}/configurations?api-version={api-version} |
HTTP 200 (OK) on successful completion of the operation.
Example response:
"configurations":
{
"gateway": {
"restAuthCredential.isEnabled": true,
"restAuthCredential.username": "username",
"restAuthCredential.password": "Password123!"
},
"core-site": {
"fs.defaultFS": "wasbs://mycontainter@mystorageaccount.blob.core.windows.net",
"fs.azure.account.key.mystorageaccount.blob.core.windows.net": "storagekey1value"
}
}
Element name | Type | Description |
---|---|---|
configurations | Dictionary | This element is a dictionary of configuration type and its associated value dictionary. gateway configuration type is used to configure the http user used for connecting to web APIs and the Ambari portal core-site configuration type is used to configure the default storage account for the cluster |
Gets details about a single configuration type (omits sensitive values).
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
GET | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}/configurations/{configuration Type}?api-version={api-version} |
HTTP 200 (OK) on successful completion of the operation.
Example response:
{
"core-site": {
"fs.defaultFS": "wasbs://mycontainter@mystorageaccount.blob.core.windows.net",
"fs.azure.account.key.mystorageaccount.blob.core.windows.net": ""
}
}
This operation retrieves cluster gateway HTTP credentials.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
POST | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}/getGatewaySettings?api-version={api-version} |
HTTP 200 (OK) on successful completion of the operation.
Example response:
{
"restAuthCredential.isEnabled": true,
"restAuthCredential.username": "username",
"restAuthCredential.password": "Password123!"
}
This operation allows users to change the cluster gateway HTTP credentials.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
POST | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}/updateGatewaySettings?api-version={api-version} |
{
"restAuthCredential.isEnabled": true,
"restAuthCredential.username": "username",
"restAuthCredential.password": "Password123!"
}
HTTP 202 (Accepted) on successful completion of the operation.
This operation allows users to enable/disable the HTTPS connectivity to the cluster.
Use updateGatewaySettings instead.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
POST | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}/configurations/{configuration Type}?api-version={api-version} |
To enable connectivity
{
"restAuthCredential.isEnabled": true,
"restAuthCredential.username": "user",
"restAuthCredential.password": "password here"
}
Lists all the clusters in the user’s subscription in the specified resource group.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
GET | https://management.azure.com/subscriptions/{subscription Id}/resourcegroups/{resourceGroup Id}/providers/Microsoft.HDInsight/clusters?api-version={api-version} |
The operation will return 200 (OK) if the request is completed successfully.
Status code: 200 OK
Response body is an array of cluster details. For more information, see Get cluster properties.
{
"value": [
{ Cluster details }
]
}
Lists all the clusters in the user’s subscription.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
GET | https://management.azure.com/subscriptions/{subscription Id}/providers/Microsoft.HDInsight/clusters?api-version={api-version} |
The operation will return 200 (OK) if the request is completed successfully.
Status code: 200 OK
Response body is an array of cluster details.
{
"value": [
{ Cluster details }
]
}
To disable connectivity
{
"restAuthCredential.isEnabled": false
}
Element name | Required | Type | Description |
---|---|---|---|
restAuthCredential.isEnabled | Yes | Boolean | Specifies if connectivity should be enabled or disabled |
restAuthCredential.username | No | String | Required if isEnabled=true Specifies the username for connectivity settings |
restAuthCredential.password | No | String | Required if isEnabled=true Specifies the password for connectivity settings |
HTTP 202 (Accepted) to indicate that the operation will complete asynchronously. Async polling will return a 204 (NoContent) once the operation completes successfully.
To track progress of a delete cluster request, see Asynchronous Operations (202 Accepted and Location header)
This operation allows a user to enable/disable RDP. It applies to Windows-based clusters.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
POST | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}/changerdpsetting?api-version={api-version} |
Request Body
To enable RDP
{
"osProfile": {
"windowsOperatingSystemProfile": {
"rdpSettings": {
"username": "username",
"password": "password here",
"expiryDate": "YYYY-MM-DD"
}
}
}
}
To disable RDP
{
"osProfile": {
"windowsOperatingSystemProfile": {
"rdpSettings": null
}
}
}
Element name | Required | Type | Description |
---|---|---|---|
windowsOperatingSystemProfile | No | Complex Type | Specifies windows OS-related settings |
Element name | Required | Type | Description |
---|---|---|---|
rdpSettings | No | Complex Type | Specifies RDP settings for windows clusters |
Element name | Required | Type | Description |
---|---|---|---|
username | Yes | String | Specifies the RDP user name |
password | Yes | String | Specifies the password for the RDP user |
expiryDate | Yes | Date | Expiry date for the RDP credentials |
The operation will return 200 (OK) if the request is completed successfully
Status code: 200 OK
Response body is the same as Create a cluster.
Execute Script action on a running cluster.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
POST | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}/executeScriptActions?api-version={api-version} |
Request Body
{
"scriptActions": [
{
"name": "script-name",
"uri": "script-uri",
"parameters": "script-parameters",
"roles": [
"headnode",
"workernode"
]
},
...
],
"persistOnSuccess": true
}
Element name | Required | Type | Description |
---|---|---|---|
name | Yes | String | Specifies the name of the script action |
uri | Yes | String | Specifies the URI of the script action |
parameters | Yes | String | Specifies the parameters required by the script action |
roles | Yes | Array of String | Specifies the target roles that the script action executes on |
persistOnSuccess | Yes | Boolean | Specifies whether the script actions will be persisted after successful executions |
If validation is complete and the request is accepted, the operation will return 202 (Accepted).
Status code: 202 (Accepted)
This operation returns all the persisted scripts actions of the specified cluster.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
GET | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}/scriptActions?api-version={api-version} |
HTTP 200 (OK) on successful completion of the operation.
Status code: 200 OK
Example response:
{
"value":
[
{
"name":"script-name",
"uri":"script-uri",
"parameters":"script-parameters",
"roles":["headnode","workernode"],
"applicationName":null
},
...
]
}
Element name | Type | Description |
---|---|---|
name | String | Specifies the name of the script action. |
uri | String | Specifies the URI of the script action. |
parameters | String | Specifies the parameters required by the script action |
roles | Array of String | Specifies the targeted roles that the script action executes on. |
applicationName | String | Specifies the corresponding application that the script is associated with. applicationName is null if the script is provided by users |
This operation removes an HDInsight persisted script action for a cluster.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
DELETE | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}/scriptActions/{script Name}?api-version={api-version} |
HTTP 200 (OK) to indicate that the script action has been removed from the list of persisted script actions.
HTTP 404 (NotFound) to indicate that there is no existing persisted script action with corresponding scriptName.
This operation returns latest scripts action execution of the specified cluster or execution details for an individual script execution.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
GET | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}/scriptExecutionHistory/{script Execution Id}?api-version={api-version} |
The operation will return 200 (OK) if the request is completed successfully. Response body is an array of script execution details or a single script execution details if scriptExecutionId is provided. Below is an example of a script execution detail.
Status code: 200 (OK)
Example response:
{
"scriptExecutionId":script-execution-id,
"name":"script-name",
"applicationName":null,
"uri":"script-uri",
"parameters":"script-parameters",
"roles":["headnode","workernode"],
"startTime":"2016-02-26T23:49:13.0773637Z",
"endTime":"2016-02-26T23:49:33.4964725Z",
"status":"Succeeded",
"operation":"PostClusterCreateScriptActionRequest",
"executionSummary":
[{"status":"COMPLETED",
"instanceCount":4}],
"debugInformation": "debug-information"
}
Element name | Type | Description |
---|---|---|
scriptExecutionId | Long | Specifies the execution id of the script action. |
name | String | Specifies the name of the script action. |
applicationName | String | Specifies the corresponding application that the script is associated with. applicationName is null if the script is provided by users |
uri | String | Specifies the URI of the script action. |
parameters | String | Specifies the parameters required by the script action |
roles | Array of String | Specifies the targeted roles that the script action executes on. |
startTime | DateTime | Specifies the start time of the script action execution |
endTime | DateTime | Specifies the end time of the script action execution |
status | String | Specifies the status of the script action execution |
operation | String | Specifies the reason why the script action was executed. For example, ScaleUp means that the script action was executed during cluster scale up. |
executionSummary | Array of complex type | Specifies the summary of execution in terms of how many hosts succeeded and how many hosts failed to execute the script. |
debugInformation | String | Specifies detailed debug information for the script. debugInformation is returned only when a scriptExecutionId is provided in the request. |
Element name | Type | Description |
---|---|---|
status | String | Specifies the status of the execution on individual hosts. |
instanceCount | Int | Specifies the number of executions with corresponding status. |
This operation promotes an HDInsight script from script execution history to be persisted.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
POST | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}/scriptExecutionHistory/{script Execution Id}/promote?api-version={api-version} |
HTTP 200 (OK) to indicate that the script action has been promoted.
HTTP 404 (NotFound) to indicate that there is no existing script action execution with corresponding scriptExecutionId.
Resizes an existing HDInsight cluster.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
POST | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}/roles/{role name}/resize?api-version={api-version} |
Following shows an example request to update tags for a cluster
{
"targetInstanceCount": 10
}
Element name | Required | Type | Description |
---|---|---|---|
targetInstanceCount | Yes | Int | Specifies the new instance count for the role |
HTTP 202 (Accepted) to indicate that the operation will complete asynchronously. Async polling will return a 204 (NoContent) once the operation completes successfully.
To track progress of a delete cluster request, see Asynchronous Operations (202 Accepted and Location header)
Updates tags for a cluster.
All other updates are separate actions described separately.
See Common parameters and headers for headers and parameters that are used by clusters.
Method | Request URI |
---|---|
PATCH | https://management.azure.com/subscriptions/{subscription Id}/resourceGroups/{resourceGroup Name}/providers/Microsoft.HDInsight/clusters/{cluster name}?api-version={api-version} |
Following shows an example request to update tags for a cluster
{
"tags": {"department": "finance"}
}
Element name | Required | Type | Description |
---|---|---|---|
tags | Yes | String | Specifies the tags that will be assigned to the cluster. For more information about using tags, see Using tags to organize your Azure resources. |
The operation will return 200 (OK) if the request is completed successfully.
Status code: 200 OK
Response body is the same as Create a cluster.