Microsoft.ContainerService managedClusters/agentPools 2022-03-01
- Latest
- 2024-06-02-preview
- 2024-05-02-preview
- 2024-05-01
- 2024-04-02-preview
- 2024-03-02-preview
- 2024-02-01
- 2024-01-02-preview
- 2024-01-01
- 2023-11-02-preview
- 2023-11-01
- 2023-10-02-preview
- 2023-10-01
- 2023-09-02-preview
- 2023-09-01
- 2023-08-02-preview
- 2023-08-01
- 2023-07-02-preview
- 2023-07-01
- 2023-06-02-preview
- 2023-06-01
- 2023-05-02-preview
- 2023-05-01
- 2023-04-02-preview
- 2023-04-01
- 2023-03-02-preview
- 2023-03-01
- 2023-02-02-preview
- 2023-02-01
- 2023-01-02-preview
- 2023-01-01
- 2022-11-02-preview
- 2022-11-01
- 2022-10-02-preview
- 2022-09-01
- 2022-08-03-preview
- 2022-08-02-preview
- 2022-07-01
- 2022-06-01
- 2022-04-01
- 2022-03-01
- 2022-02-01
- 2022-01-01
- 2021-10-01
- 2021-09-01
- 2021-08-01
- 2021-07-01
- 2021-05-01
- 2021-03-01
- 2021-02-01
- 2020-12-01
- 2020-11-01
- 2020-09-01
Bicep resource definition
The managedClusters/agentPools resource type can be deployed with operations that target:
- Resource groups - See resource group deployment commands
For a list of changed properties in each API version, see change log.
Resource format
To create a Microsoft.ContainerService/managedClusters/agentPools resource, add the following Bicep to your template.
resource symbolicname 'Microsoft.ContainerService/managedClusters/agentPools@2022-03-01' = {
name: 'string'
parent: resourceSymbolicName
properties: {
availabilityZones: [
'string'
]
count: int
creationData: {
sourceResourceId: 'string'
}
enableAutoScaling: bool
enableEncryptionAtHost: bool
enableFIPS: bool
enableNodePublicIP: bool
enableUltraSSD: bool
gpuInstanceProfile: 'string'
kubeletConfig: {
allowedUnsafeSysctls: [
'string'
]
containerLogMaxFiles: int
containerLogMaxSizeMB: int
cpuCfsQuota: bool
cpuCfsQuotaPeriod: 'string'
cpuManagerPolicy: 'string'
failSwapOn: bool
imageGcHighThreshold: int
imageGcLowThreshold: int
podMaxPids: int
topologyManagerPolicy: 'string'
}
kubeletDiskType: 'string'
linuxOSConfig: {
swapFileSizeMB: int
sysctls: {
fsAioMaxNr: int
fsFileMax: int
fsInotifyMaxUserWatches: int
fsNrOpen: int
kernelThreadsMax: int
netCoreNetdevMaxBacklog: int
netCoreOptmemMax: int
netCoreRmemDefault: int
netCoreRmemMax: int
netCoreSomaxconn: int
netCoreWmemDefault: int
netCoreWmemMax: int
netIpv4IpLocalPortRange: 'string'
netIpv4NeighDefaultGcThresh1: int
netIpv4NeighDefaultGcThresh2: int
netIpv4NeighDefaultGcThresh3: int
netIpv4TcpFinTimeout: int
netIpv4TcpkeepaliveIntvl: int
netIpv4TcpKeepaliveProbes: int
netIpv4TcpKeepaliveTime: int
netIpv4TcpMaxSynBacklog: int
netIpv4TcpMaxTwBuckets: int
netIpv4TcpTwReuse: bool
netNetfilterNfConntrackBuckets: int
netNetfilterNfConntrackMax: int
vmMaxMapCount: int
vmSwappiness: int
vmVfsCachePressure: int
}
transparentHugePageDefrag: 'string'
transparentHugePageEnabled: 'string'
}
maxCount: int
maxPods: int
minCount: int
mode: 'string'
nodeLabels: {
{customized property}: 'string'
}
nodePublicIPPrefixID: 'string'
nodeTaints: [
'string'
]
orchestratorVersion: 'string'
osDiskSizeGB: int
osDiskType: 'string'
osSKU: 'string'
osType: 'string'
podSubnetID: 'string'
powerState: {
code: 'string'
}
proximityPlacementGroupID: 'string'
scaleDownMode: 'string'
scaleSetEvictionPolicy: 'string'
scaleSetPriority: 'string'
spotMaxPrice: int
tags: {}
type: 'string'
upgradeSettings: {
maxSurge: 'string'
}
vmSize: 'string'
vnetSubnetID: 'string'
workloadRuntime: 'string'
}
}
Property values
managedClusters/agentPools
| Name | Description | Value |
|---|---|---|
| name | The resource name See how to set names and types for child resources in Bicep. |
string (required) Character limit: 1-12 for Linux 1-6 for Windows Valid characters: Lowercase letters and numbers. Can't start with a number. |
| parent | In Bicep, you can specify the parent resource for a child resource. You only need to add this property when the child resource is declared outside of the parent resource. For more information, see Child resource outside parent resource. |
Symbolic name for resource of type: managedClusters |
| properties | Properties of an agent pool. | ManagedClusterAgentPoolProfileProperties |
ManagedClusterAgentPoolProfileProperties
| Name | Description | Value |
|---|---|---|
| availabilityZones | The list of Availability zones to use for nodes. This can only be specified if the AgentPoolType property is 'VirtualMachineScaleSets'. | string[] |
| count | Number of agents (VMs) to host docker containers. Allowed values must be in the range of 0 to 1000 (inclusive) for user pools and in the range of 1 to 1000 (inclusive) for system pools. The default value is 1. | int |
| creationData | CreationData to be used to specify the source Snapshot ID if the node pool will be created/upgraded using a snapshot. | CreationData |
| enableAutoScaling | Whether to enable auto-scaler | bool |
| enableEncryptionAtHost | This is only supported on certain VM sizes and in certain Azure regions. For more information, see: /azure/aks/enable-host-encryption | bool |
| enableFIPS | See Add a FIPS-enabled node pool for more details. | bool |
| enableNodePublicIP | Some scenarios may require nodes in a node pool to receive their own dedicated public IP addresses. A common scenario is for gaming workloads, where a console needs to make a direct connection to a cloud virtual machine to minimize hops. For more information see assigning a public IP per node. The default is false. | bool |
| enableUltraSSD | Whether to enable UltraSSD | bool |
| gpuInstanceProfile | GPUInstanceProfile to be used to specify GPU MIG instance profile for supported GPU VM SKU. | 'MIG1g' 'MIG2g' 'MIG3g' 'MIG4g' 'MIG7g' |
| kubeletConfig | The Kubelet configuration on the agent pool nodes. | KubeletConfig |
| kubeletDiskType | Determines the placement of emptyDir volumes, container runtime data root, and Kubelet ephemeral storage. | 'OS' 'Temporary' |
| linuxOSConfig | The OS configuration of Linux agent nodes. | LinuxOSConfig |
| maxCount | The maximum number of nodes for auto-scaling | int |
| maxPods | The maximum number of pods that can run on a node. | int |
| minCount | The minimum number of nodes for auto-scaling | int |
| mode | A cluster must have at least one 'System' Agent Pool at all times. For additional information on agent pool restrictions and best practices, see: /azure/aks/use-system-pools | 'System' 'User' |
| nodeLabels | The node labels to be persisted across all nodes in agent pool. | ManagedClusterAgentPoolProfilePropertiesNodeLabels |
| nodePublicIPPrefixID | This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/publicIPPrefixes/{publicIPPrefixName} | string |
| nodeTaints | The taints added to new nodes during node pool create and scale. For example, key=value:NoSchedule. | string[] |
| orchestratorVersion | As a best practice, you should upgrade all node pools in an AKS cluster to the same Kubernetes version. The node pool version must have the same major version as the control plane. The node pool minor version must be within two minor versions of the control plane version. The node pool version cannot be greater than the control plane version. For more information see upgrading a node pool. | string |
| osDiskSizeGB | OS Disk Size in GB to be used to specify the disk size for every machine in the master/agent pool. If you specify 0, it will apply the default osDisk size according to the vmSize specified. | int Constraints: Min value = 0 Max value = 2048 |
| osDiskType | The default is 'Ephemeral' if the VM supports it and has a cache disk larger than the requested OSDiskSizeGB. Otherwise, defaults to 'Managed'. May not be changed after creation. For more information see Ephemeral OS. | 'Ephemeral' 'Managed' |
| osSKU | Specifies an OS SKU. This value must not be specified if OSType is Windows. | 'CBLMariner' 'Ubuntu' |
| osType | The operating system type. The default is Linux. | 'Linux' 'Windows' |
| podSubnetID | If omitted, pod IPs are statically assigned on the node subnet (see vnetSubnetID for more details). This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName} | string |
| powerState | When an Agent Pool is first created it is initially Running. The Agent Pool can be stopped by setting this field to Stopped. A stopped Agent Pool stops all of its VMs and does not accrue billing charges. An Agent Pool can only be stopped if it is Running and provisioning state is Succeeded | PowerState |
| proximityPlacementGroupID | The ID for Proximity Placement Group. | string |
| scaleDownMode | This also effects the cluster autoscaler behavior. If not specified, it defaults to Delete. | 'Deallocate' 'Delete' |
| scaleSetEvictionPolicy | This cannot be specified unless the scaleSetPriority is 'Spot'. If not specified, the default is 'Delete'. | 'Deallocate' 'Delete' |
| scaleSetPriority | The Virtual Machine Scale Set priority. If not specified, the default is 'Regular'. | 'Regular' 'Spot' |
| spotMaxPrice | Possible values are any decimal value greater than zero or -1 which indicates the willingness to pay any on-demand price. For more details on spot pricing, see spot VMs pricing | int |
| tags | The tags to be persisted on the agent pool virtual machine scale set. | object |
| type | The type of Agent Pool. | 'AvailabilitySet' 'VirtualMachineScaleSets' |
| upgradeSettings | Settings for upgrading the agentpool | AgentPoolUpgradeSettings |
| vmSize | VM size availability varies by region. If a node contains insufficient compute resources (memory, cpu, etc) pods might fail to run correctly. For more details on restricted VM sizes, see: /azure/aks/quotas-skus-regions | string |
| vnetSubnetID | If this is not specified, a VNET and subnet will be generated and used. If no podSubnetID is specified, this applies to nodes and pods, otherwise it applies to just nodes. This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName} | string |
| workloadRuntime | Determines the type of workload a node can run. | 'OCIContainer' 'WasmWasi' |
CreationData
| Name | Description | Value |
|---|---|---|
| sourceResourceId | This is the ARM ID of the source object to be used to create the target object. | string |
KubeletConfig
| Name | Description | Value |
|---|---|---|
| allowedUnsafeSysctls | Allowed list of unsafe sysctls or unsafe sysctl patterns (ending in *). |
string[] |
| containerLogMaxFiles | The maximum number of container log files that can be present for a container. The number must be ≥ 2. | int Constraints: Min value = 2 |
| containerLogMaxSizeMB | The maximum size (e.g. 10Mi) of container log file before it is rotated. | int |
| cpuCfsQuota | The default is true. | bool |
| cpuCfsQuotaPeriod | The default is '100ms.' Valid values are a sequence of decimal numbers with an optional fraction and a unit suffix. For example: '300ms', '2h45m'. Supported units are 'ns', 'us', 'ms', 's', 'm', and 'h'. | string |
| cpuManagerPolicy | The default is 'none'. See Kubernetes CPU management policies for more information. Allowed values are 'none' and 'static'. | string |
| failSwapOn | If set to true it will make the Kubelet fail to start if swap is enabled on the node. | bool |
| imageGcHighThreshold | To disable image garbage collection, set to 100. The default is 85% | int |
| imageGcLowThreshold | This cannot be set higher than imageGcHighThreshold. The default is 80% | int |
| podMaxPids | The maximum number of processes per pod. | int |
| topologyManagerPolicy | For more information see Kubernetes Topology Manager. The default is 'none'. Allowed values are 'none', 'best-effort', 'restricted', and 'single-numa-node'. | string |
LinuxOSConfig
| Name | Description | Value |
|---|---|---|
| swapFileSizeMB | The size in MB of a swap file that will be created on each node. | int |
| sysctls | Sysctl settings for Linux agent nodes. | SysctlConfig |
| transparentHugePageDefrag | Valid values are 'always', 'defer', 'defer+madvise', 'madvise' and 'never'. The default is 'madvise'. For more information see Transparent Hugepages. | string |
| transparentHugePageEnabled | Valid values are 'always', 'madvise', and 'never'. The default is 'always'. For more information see Transparent Hugepages. | string |
SysctlConfig
| Name | Description | Value |
|---|---|---|
| fsAioMaxNr | Sysctl setting fs.aio-max-nr. | int |
| fsFileMax | Sysctl setting fs.file-max. | int |
| fsInotifyMaxUserWatches | Sysctl setting fs.inotify.max_user_watches. | int |
| fsNrOpen | Sysctl setting fs.nr_open. | int |
| kernelThreadsMax | Sysctl setting kernel.threads-max. | int |
| netCoreNetdevMaxBacklog | Sysctl setting net.core.netdev_max_backlog. | int |
| netCoreOptmemMax | Sysctl setting net.core.optmem_max. | int |
| netCoreRmemDefault | Sysctl setting net.core.rmem_default. | int |
| netCoreRmemMax | Sysctl setting net.core.rmem_max. | int |
| netCoreSomaxconn | Sysctl setting net.core.somaxconn. | int |
| netCoreWmemDefault | Sysctl setting net.core.wmem_default. | int |
| netCoreWmemMax | Sysctl setting net.core.wmem_max. | int |
| netIpv4IpLocalPortRange | Sysctl setting net.ipv4.ip_local_port_range. | string |
| netIpv4NeighDefaultGcThresh1 | Sysctl setting net.ipv4.neigh.default.gc_thresh1. | int |
| netIpv4NeighDefaultGcThresh2 | Sysctl setting net.ipv4.neigh.default.gc_thresh2. | int |
| netIpv4NeighDefaultGcThresh3 | Sysctl setting net.ipv4.neigh.default.gc_thresh3. | int |
| netIpv4TcpFinTimeout | Sysctl setting net.ipv4.tcp_fin_timeout. | int |
| netIpv4TcpkeepaliveIntvl | Sysctl setting net.ipv4.tcp_keepalive_intvl. | int |
| netIpv4TcpKeepaliveProbes | Sysctl setting net.ipv4.tcp_keepalive_probes. | int |
| netIpv4TcpKeepaliveTime | Sysctl setting net.ipv4.tcp_keepalive_time. | int |
| netIpv4TcpMaxSynBacklog | Sysctl setting net.ipv4.tcp_max_syn_backlog. | int |
| netIpv4TcpMaxTwBuckets | Sysctl setting net.ipv4.tcp_max_tw_buckets. | int |
| netIpv4TcpTwReuse | Sysctl setting net.ipv4.tcp_tw_reuse. | bool |
| netNetfilterNfConntrackBuckets | Sysctl setting net.netfilter.nf_conntrack_buckets. | int |
| netNetfilterNfConntrackMax | Sysctl setting net.netfilter.nf_conntrack_max. | int |
| vmMaxMapCount | Sysctl setting vm.max_map_count. | int |
| vmSwappiness | Sysctl setting vm.swappiness. | int |
| vmVfsCachePressure | Sysctl setting vm.vfs_cache_pressure. | int |
ManagedClusterAgentPoolProfilePropertiesNodeLabels
| Name | Description | Value |
|---|---|---|
| {customized property} | string |
PowerState
| Name | Description | Value |
|---|---|---|
| code | Tells whether the cluster is Running or Stopped | 'Running' 'Stopped' |
AgentPoolUpgradeSettings
| Name | Description | Value |
|---|---|---|
| maxSurge | This can either be set to an integer (e.g. '5') or a percentage (e.g. '50%'). If a percentage is specified, it is the percentage of the total agent pool size at the time of the upgrade. For percentages, fractional nodes are rounded up. If not specified, the default is 1. For more information, including best practices, see: /azure/aks/upgrade-cluster#customize-node-surge-upgrade | string |
Quickstart templates
The following quickstart templates deploy this resource type.
| Template | Description |
|---|---|
Deploy an AKS cluster for Azure ML |
This template allows you to deploy an entreprise compliant AKS cluster which can be attached to Azure ML |
ARM template resource definition
The managedClusters/agentPools resource type can be deployed with operations that target:
- Resource groups - See resource group deployment commands
For a list of changed properties in each API version, see change log.
Resource format
To create a Microsoft.ContainerService/managedClusters/agentPools resource, add the following JSON to your template.
{
"type": "Microsoft.ContainerService/managedClusters/agentPools",
"apiVersion": "2022-03-01",
"name": "string",
"properties": {
"availabilityZones": [ "string" ],
"count": "int",
"creationData": {
"sourceResourceId": "string"
},
"enableAutoScaling": "bool",
"enableEncryptionAtHost": "bool",
"enableFIPS": "bool",
"enableNodePublicIP": "bool",
"enableUltraSSD": "bool",
"gpuInstanceProfile": "string",
"kubeletConfig": {
"allowedUnsafeSysctls": [ "string" ],
"containerLogMaxFiles": "int",
"containerLogMaxSizeMB": "int",
"cpuCfsQuota": "bool",
"cpuCfsQuotaPeriod": "string",
"cpuManagerPolicy": "string",
"failSwapOn": "bool",
"imageGcHighThreshold": "int",
"imageGcLowThreshold": "int",
"podMaxPids": "int",
"topologyManagerPolicy": "string"
},
"kubeletDiskType": "string",
"linuxOSConfig": {
"swapFileSizeMB": "int",
"sysctls": {
"fsAioMaxNr": "int",
"fsFileMax": "int",
"fsInotifyMaxUserWatches": "int",
"fsNrOpen": "int",
"kernelThreadsMax": "int",
"netCoreNetdevMaxBacklog": "int",
"netCoreOptmemMax": "int",
"netCoreRmemDefault": "int",
"netCoreRmemMax": "int",
"netCoreSomaxconn": "int",
"netCoreWmemDefault": "int",
"netCoreWmemMax": "int",
"netIpv4IpLocalPortRange": "string",
"netIpv4NeighDefaultGcThresh1": "int",
"netIpv4NeighDefaultGcThresh2": "int",
"netIpv4NeighDefaultGcThresh3": "int",
"netIpv4TcpFinTimeout": "int",
"netIpv4TcpkeepaliveIntvl": "int",
"netIpv4TcpKeepaliveProbes": "int",
"netIpv4TcpKeepaliveTime": "int",
"netIpv4TcpMaxSynBacklog": "int",
"netIpv4TcpMaxTwBuckets": "int",
"netIpv4TcpTwReuse": "bool",
"netNetfilterNfConntrackBuckets": "int",
"netNetfilterNfConntrackMax": "int",
"vmMaxMapCount": "int",
"vmSwappiness": "int",
"vmVfsCachePressure": "int"
},
"transparentHugePageDefrag": "string",
"transparentHugePageEnabled": "string"
},
"maxCount": "int",
"maxPods": "int",
"minCount": "int",
"mode": "string",
"nodeLabels": {
"{customized property}": "string"
},
"nodePublicIPPrefixID": "string",
"nodeTaints": [ "string" ],
"orchestratorVersion": "string",
"osDiskSizeGB": "int",
"osDiskType": "string",
"osSKU": "string",
"osType": "string",
"podSubnetID": "string",
"powerState": {
"code": "string"
},
"proximityPlacementGroupID": "string",
"scaleDownMode": "string",
"scaleSetEvictionPolicy": "string",
"scaleSetPriority": "string",
"spotMaxPrice": "int",
"tags": {},
"type": "string",
"upgradeSettings": {
"maxSurge": "string"
},
"vmSize": "string",
"vnetSubnetID": "string",
"workloadRuntime": "string"
}
}
Property values
managedClusters/agentPools
| Name | Description | Value |
|---|---|---|
| type | The resource type | 'Microsoft.ContainerService/managedClusters/agentPools' |
| apiVersion | The resource api version | '2022-03-01' |
| name | The resource name See how to set names and types for child resources in JSON ARM templates. |
string (required) Character limit: 1-12 for Linux 1-6 for Windows Valid characters: Lowercase letters and numbers. Can't start with a number. |
| properties | Properties of an agent pool. | ManagedClusterAgentPoolProfileProperties |
ManagedClusterAgentPoolProfileProperties
| Name | Description | Value |
|---|---|---|
| availabilityZones | The list of Availability zones to use for nodes. This can only be specified if the AgentPoolType property is 'VirtualMachineScaleSets'. | string[] |
| count | Number of agents (VMs) to host docker containers. Allowed values must be in the range of 0 to 1000 (inclusive) for user pools and in the range of 1 to 1000 (inclusive) for system pools. The default value is 1. | int |
| creationData | CreationData to be used to specify the source Snapshot ID if the node pool will be created/upgraded using a snapshot. | CreationData |
| enableAutoScaling | Whether to enable auto-scaler | bool |
| enableEncryptionAtHost | This is only supported on certain VM sizes and in certain Azure regions. For more information, see: /azure/aks/enable-host-encryption | bool |
| enableFIPS | See Add a FIPS-enabled node pool for more details. | bool |
| enableNodePublicIP | Some scenarios may require nodes in a node pool to receive their own dedicated public IP addresses. A common scenario is for gaming workloads, where a console needs to make a direct connection to a cloud virtual machine to minimize hops. For more information see assigning a public IP per node. The default is false. | bool |
| enableUltraSSD | Whether to enable UltraSSD | bool |
| gpuInstanceProfile | GPUInstanceProfile to be used to specify GPU MIG instance profile for supported GPU VM SKU. | 'MIG1g' 'MIG2g' 'MIG3g' 'MIG4g' 'MIG7g' |
| kubeletConfig | The Kubelet configuration on the agent pool nodes. | KubeletConfig |
| kubeletDiskType | Determines the placement of emptyDir volumes, container runtime data root, and Kubelet ephemeral storage. | 'OS' 'Temporary' |
| linuxOSConfig | The OS configuration of Linux agent nodes. | LinuxOSConfig |
| maxCount | The maximum number of nodes for auto-scaling | int |
| maxPods | The maximum number of pods that can run on a node. | int |
| minCount | The minimum number of nodes for auto-scaling | int |
| mode | A cluster must have at least one 'System' Agent Pool at all times. For additional information on agent pool restrictions and best practices, see: /azure/aks/use-system-pools | 'System' 'User' |
| nodeLabels | The node labels to be persisted across all nodes in agent pool. | ManagedClusterAgentPoolProfilePropertiesNodeLabels |
| nodePublicIPPrefixID | This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/publicIPPrefixes/{publicIPPrefixName} | string |
| nodeTaints | The taints added to new nodes during node pool create and scale. For example, key=value:NoSchedule. | string[] |
| orchestratorVersion | As a best practice, you should upgrade all node pools in an AKS cluster to the same Kubernetes version. The node pool version must have the same major version as the control plane. The node pool minor version must be within two minor versions of the control plane version. The node pool version cannot be greater than the control plane version. For more information see upgrading a node pool. | string |
| osDiskSizeGB | OS Disk Size in GB to be used to specify the disk size for every machine in the master/agent pool. If you specify 0, it will apply the default osDisk size according to the vmSize specified. | int Constraints: Min value = 0 Max value = 2048 |
| osDiskType | The default is 'Ephemeral' if the VM supports it and has a cache disk larger than the requested OSDiskSizeGB. Otherwise, defaults to 'Managed'. May not be changed after creation. For more information see Ephemeral OS. | 'Ephemeral' 'Managed' |
| osSKU | Specifies an OS SKU. This value must not be specified if OSType is Windows. | 'CBLMariner' 'Ubuntu' |
| osType | The operating system type. The default is Linux. | 'Linux' 'Windows' |
| podSubnetID | If omitted, pod IPs are statically assigned on the node subnet (see vnetSubnetID for more details). This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName} | string |
| powerState | When an Agent Pool is first created it is initially Running. The Agent Pool can be stopped by setting this field to Stopped. A stopped Agent Pool stops all of its VMs and does not accrue billing charges. An Agent Pool can only be stopped if it is Running and provisioning state is Succeeded | PowerState |
| proximityPlacementGroupID | The ID for Proximity Placement Group. | string |
| scaleDownMode | This also effects the cluster autoscaler behavior. If not specified, it defaults to Delete. | 'Deallocate' 'Delete' |
| scaleSetEvictionPolicy | This cannot be specified unless the scaleSetPriority is 'Spot'. If not specified, the default is 'Delete'. | 'Deallocate' 'Delete' |
| scaleSetPriority | The Virtual Machine Scale Set priority. If not specified, the default is 'Regular'. | 'Regular' 'Spot' |
| spotMaxPrice | Possible values are any decimal value greater than zero or -1 which indicates the willingness to pay any on-demand price. For more details on spot pricing, see spot VMs pricing | int |
| tags | The tags to be persisted on the agent pool virtual machine scale set. | object |
| type | The type of Agent Pool. | 'AvailabilitySet' 'VirtualMachineScaleSets' |
| upgradeSettings | Settings for upgrading the agentpool | AgentPoolUpgradeSettings |
| vmSize | VM size availability varies by region. If a node contains insufficient compute resources (memory, cpu, etc) pods might fail to run correctly. For more details on restricted VM sizes, see: /azure/aks/quotas-skus-regions | string |
| vnetSubnetID | If this is not specified, a VNET and subnet will be generated and used. If no podSubnetID is specified, this applies to nodes and pods, otherwise it applies to just nodes. This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName} | string |
| workloadRuntime | Determines the type of workload a node can run. | 'OCIContainer' 'WasmWasi' |
CreationData
| Name | Description | Value |
|---|---|---|
| sourceResourceId | This is the ARM ID of the source object to be used to create the target object. | string |
KubeletConfig
| Name | Description | Value |
|---|---|---|
| allowedUnsafeSysctls | Allowed list of unsafe sysctls or unsafe sysctl patterns (ending in *). |
string[] |
| containerLogMaxFiles | The maximum number of container log files that can be present for a container. The number must be ≥ 2. | int Constraints: Min value = 2 |
| containerLogMaxSizeMB | The maximum size (e.g. 10Mi) of container log file before it is rotated. | int |
| cpuCfsQuota | The default is true. | bool |
| cpuCfsQuotaPeriod | The default is '100ms.' Valid values are a sequence of decimal numbers with an optional fraction and a unit suffix. For example: '300ms', '2h45m'. Supported units are 'ns', 'us', 'ms', 's', 'm', and 'h'. | string |
| cpuManagerPolicy | The default is 'none'. See Kubernetes CPU management policies for more information. Allowed values are 'none' and 'static'. | string |
| failSwapOn | If set to true it will make the Kubelet fail to start if swap is enabled on the node. | bool |
| imageGcHighThreshold | To disable image garbage collection, set to 100. The default is 85% | int |
| imageGcLowThreshold | This cannot be set higher than imageGcHighThreshold. The default is 80% | int |
| podMaxPids | The maximum number of processes per pod. | int |
| topologyManagerPolicy | For more information see Kubernetes Topology Manager. The default is 'none'. Allowed values are 'none', 'best-effort', 'restricted', and 'single-numa-node'. | string |
LinuxOSConfig
| Name | Description | Value |
|---|---|---|
| swapFileSizeMB | The size in MB of a swap file that will be created on each node. | int |
| sysctls | Sysctl settings for Linux agent nodes. | SysctlConfig |
| transparentHugePageDefrag | Valid values are 'always', 'defer', 'defer+madvise', 'madvise' and 'never'. The default is 'madvise'. For more information see Transparent Hugepages. | string |
| transparentHugePageEnabled | Valid values are 'always', 'madvise', and 'never'. The default is 'always'. For more information see Transparent Hugepages. | string |
SysctlConfig
| Name | Description | Value |
|---|---|---|
| fsAioMaxNr | Sysctl setting fs.aio-max-nr. | int |
| fsFileMax | Sysctl setting fs.file-max. | int |
| fsInotifyMaxUserWatches | Sysctl setting fs.inotify.max_user_watches. | int |
| fsNrOpen | Sysctl setting fs.nr_open. | int |
| kernelThreadsMax | Sysctl setting kernel.threads-max. | int |
| netCoreNetdevMaxBacklog | Sysctl setting net.core.netdev_max_backlog. | int |
| netCoreOptmemMax | Sysctl setting net.core.optmem_max. | int |
| netCoreRmemDefault | Sysctl setting net.core.rmem_default. | int |
| netCoreRmemMax | Sysctl setting net.core.rmem_max. | int |
| netCoreSomaxconn | Sysctl setting net.core.somaxconn. | int |
| netCoreWmemDefault | Sysctl setting net.core.wmem_default. | int |
| netCoreWmemMax | Sysctl setting net.core.wmem_max. | int |
| netIpv4IpLocalPortRange | Sysctl setting net.ipv4.ip_local_port_range. | string |
| netIpv4NeighDefaultGcThresh1 | Sysctl setting net.ipv4.neigh.default.gc_thresh1. | int |
| netIpv4NeighDefaultGcThresh2 | Sysctl setting net.ipv4.neigh.default.gc_thresh2. | int |
| netIpv4NeighDefaultGcThresh3 | Sysctl setting net.ipv4.neigh.default.gc_thresh3. | int |
| netIpv4TcpFinTimeout | Sysctl setting net.ipv4.tcp_fin_timeout. | int |
| netIpv4TcpkeepaliveIntvl | Sysctl setting net.ipv4.tcp_keepalive_intvl. | int |
| netIpv4TcpKeepaliveProbes | Sysctl setting net.ipv4.tcp_keepalive_probes. | int |
| netIpv4TcpKeepaliveTime | Sysctl setting net.ipv4.tcp_keepalive_time. | int |
| netIpv4TcpMaxSynBacklog | Sysctl setting net.ipv4.tcp_max_syn_backlog. | int |
| netIpv4TcpMaxTwBuckets | Sysctl setting net.ipv4.tcp_max_tw_buckets. | int |
| netIpv4TcpTwReuse | Sysctl setting net.ipv4.tcp_tw_reuse. | bool |
| netNetfilterNfConntrackBuckets | Sysctl setting net.netfilter.nf_conntrack_buckets. | int |
| netNetfilterNfConntrackMax | Sysctl setting net.netfilter.nf_conntrack_max. | int |
| vmMaxMapCount | Sysctl setting vm.max_map_count. | int |
| vmSwappiness | Sysctl setting vm.swappiness. | int |
| vmVfsCachePressure | Sysctl setting vm.vfs_cache_pressure. | int |
ManagedClusterAgentPoolProfilePropertiesNodeLabels
| Name | Description | Value |
|---|---|---|
| {customized property} | string |
PowerState
| Name | Description | Value |
|---|---|---|
| code | Tells whether the cluster is Running or Stopped | 'Running' 'Stopped' |
AgentPoolUpgradeSettings
| Name | Description | Value |
|---|---|---|
| maxSurge | This can either be set to an integer (e.g. '5') or a percentage (e.g. '50%'). If a percentage is specified, it is the percentage of the total agent pool size at the time of the upgrade. For percentages, fractional nodes are rounded up. If not specified, the default is 1. For more information, including best practices, see: /azure/aks/upgrade-cluster#customize-node-surge-upgrade | string |
Quickstart templates
The following quickstart templates deploy this resource type.
| Template | Description |
|---|---|
| Deploy an AKS cluster for Azure ML |
This template allows you to deploy an entreprise compliant AKS cluster which can be attached to Azure ML |
Terraform (AzAPI provider) resource definition
The managedClusters/agentPools resource type can be deployed with operations that target:
- Resource groups
For a list of changed properties in each API version, see change log.
Resource format
To create a Microsoft.ContainerService/managedClusters/agentPools resource, add the following Terraform to your template.
resource "azapi_resource" "symbolicname" {
type = "Microsoft.ContainerService/managedClusters/agentPools@2022-03-01"
name = "string"
parent_id = "string"
body = jsonencode({
properties = {
availabilityZones = [
"string"
]
count = int
creationData = {
sourceResourceId = "string"
}
enableAutoScaling = bool
enableEncryptionAtHost = bool
enableFIPS = bool
enableNodePublicIP = bool
enableUltraSSD = bool
gpuInstanceProfile = "string"
kubeletConfig = {
allowedUnsafeSysctls = [
"string"
]
containerLogMaxFiles = int
containerLogMaxSizeMB = int
cpuCfsQuota = bool
cpuCfsQuotaPeriod = "string"
cpuManagerPolicy = "string"
failSwapOn = bool
imageGcHighThreshold = int
imageGcLowThreshold = int
podMaxPids = int
topologyManagerPolicy = "string"
}
kubeletDiskType = "string"
linuxOSConfig = {
swapFileSizeMB = int
sysctls = {
fsAioMaxNr = int
fsFileMax = int
fsInotifyMaxUserWatches = int
fsNrOpen = int
kernelThreadsMax = int
netCoreNetdevMaxBacklog = int
netCoreOptmemMax = int
netCoreRmemDefault = int
netCoreRmemMax = int
netCoreSomaxconn = int
netCoreWmemDefault = int
netCoreWmemMax = int
netIpv4IpLocalPortRange = "string"
netIpv4NeighDefaultGcThresh1 = int
netIpv4NeighDefaultGcThresh2 = int
netIpv4NeighDefaultGcThresh3 = int
netIpv4TcpFinTimeout = int
netIpv4TcpkeepaliveIntvl = int
netIpv4TcpKeepaliveProbes = int
netIpv4TcpKeepaliveTime = int
netIpv4TcpMaxSynBacklog = int
netIpv4TcpMaxTwBuckets = int
netIpv4TcpTwReuse = bool
netNetfilterNfConntrackBuckets = int
netNetfilterNfConntrackMax = int
vmMaxMapCount = int
vmSwappiness = int
vmVfsCachePressure = int
}
transparentHugePageDefrag = "string"
transparentHugePageEnabled = "string"
}
maxCount = int
maxPods = int
minCount = int
mode = "string"
nodeLabels = {
{customized property} = "string"
}
nodePublicIPPrefixID = "string"
nodeTaints = [
"string"
]
orchestratorVersion = "string"
osDiskSizeGB = int
osDiskType = "string"
osSKU = "string"
osType = "string"
podSubnetID = "string"
powerState = {
code = "string"
}
proximityPlacementGroupID = "string"
scaleDownMode = "string"
scaleSetEvictionPolicy = "string"
scaleSetPriority = "string"
spotMaxPrice = int
tags = {}
type = "string"
upgradeSettings = {
maxSurge = "string"
}
vmSize = "string"
vnetSubnetID = "string"
workloadRuntime = "string"
}
})
}
Property values
managedClusters/agentPools
| Name | Description | Value |
|---|---|---|
| type | The resource type | "Microsoft.ContainerService/managedClusters/agentPools@2022-03-01" |
| name | The resource name | string (required) Character limit: 1-12 for Linux 1-6 for Windows Valid characters: Lowercase letters and numbers. Can't start with a number. |
| parent_id | The ID of the resource that is the parent for this resource. | ID for resource of type: managedClusters |
| properties | Properties of an agent pool. | ManagedClusterAgentPoolProfileProperties |
ManagedClusterAgentPoolProfileProperties
| Name | Description | Value |
|---|---|---|
| availabilityZones | The list of Availability zones to use for nodes. This can only be specified if the AgentPoolType property is 'VirtualMachineScaleSets'. | string[] |
| count | Number of agents (VMs) to host docker containers. Allowed values must be in the range of 0 to 1000 (inclusive) for user pools and in the range of 1 to 1000 (inclusive) for system pools. The default value is 1. | int |
| creationData | CreationData to be used to specify the source Snapshot ID if the node pool will be created/upgraded using a snapshot. | CreationData |
| enableAutoScaling | Whether to enable auto-scaler | bool |
| enableEncryptionAtHost | This is only supported on certain VM sizes and in certain Azure regions. For more information, see: /azure/aks/enable-host-encryption | bool |
| enableFIPS | See Add a FIPS-enabled node pool for more details. | bool |
| enableNodePublicIP | Some scenarios may require nodes in a node pool to receive their own dedicated public IP addresses. A common scenario is for gaming workloads, where a console needs to make a direct connection to a cloud virtual machine to minimize hops. For more information see assigning a public IP per node. The default is false. | bool |
| enableUltraSSD | Whether to enable UltraSSD | bool |
| gpuInstanceProfile | GPUInstanceProfile to be used to specify GPU MIG instance profile for supported GPU VM SKU. | "MIG1g" "MIG2g" "MIG3g" "MIG4g" "MIG7g" |
| kubeletConfig | The Kubelet configuration on the agent pool nodes. | KubeletConfig |
| kubeletDiskType | Determines the placement of emptyDir volumes, container runtime data root, and Kubelet ephemeral storage. | "OS" "Temporary" |
| linuxOSConfig | The OS configuration of Linux agent nodes. | LinuxOSConfig |
| maxCount | The maximum number of nodes for auto-scaling | int |
| maxPods | The maximum number of pods that can run on a node. | int |
| minCount | The minimum number of nodes for auto-scaling | int |
| mode | A cluster must have at least one 'System' Agent Pool at all times. For additional information on agent pool restrictions and best practices, see: /azure/aks/use-system-pools | "System" "User" |
| nodeLabels | The node labels to be persisted across all nodes in agent pool. | ManagedClusterAgentPoolProfilePropertiesNodeLabels |
| nodePublicIPPrefixID | This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/publicIPPrefixes/{publicIPPrefixName} | string |
| nodeTaints | The taints added to new nodes during node pool create and scale. For example, key=value:NoSchedule. | string[] |
| orchestratorVersion | As a best practice, you should upgrade all node pools in an AKS cluster to the same Kubernetes version. The node pool version must have the same major version as the control plane. The node pool minor version must be within two minor versions of the control plane version. The node pool version cannot be greater than the control plane version. For more information see upgrading a node pool. | string |
| osDiskSizeGB | OS Disk Size in GB to be used to specify the disk size for every machine in the master/agent pool. If you specify 0, it will apply the default osDisk size according to the vmSize specified. | int Constraints: Min value = 0 Max value = 2048 |
| osDiskType | The default is 'Ephemeral' if the VM supports it and has a cache disk larger than the requested OSDiskSizeGB. Otherwise, defaults to 'Managed'. May not be changed after creation. For more information see Ephemeral OS. | "Ephemeral" "Managed" |
| osSKU | Specifies an OS SKU. This value must not be specified if OSType is Windows. | "CBLMariner" "Ubuntu" |
| osType | The operating system type. The default is Linux. | "Linux" "Windows" |
| podSubnetID | If omitted, pod IPs are statically assigned on the node subnet (see vnetSubnetID for more details). This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName} | string |
| powerState | When an Agent Pool is first created it is initially Running. The Agent Pool can be stopped by setting this field to Stopped. A stopped Agent Pool stops all of its VMs and does not accrue billing charges. An Agent Pool can only be stopped if it is Running and provisioning state is Succeeded | PowerState |
| proximityPlacementGroupID | The ID for Proximity Placement Group. | string |
| scaleDownMode | This also effects the cluster autoscaler behavior. If not specified, it defaults to Delete. | "Deallocate" "Delete" |
| scaleSetEvictionPolicy | This cannot be specified unless the scaleSetPriority is 'Spot'. If not specified, the default is 'Delete'. | "Deallocate" "Delete" |
| scaleSetPriority | The Virtual Machine Scale Set priority. If not specified, the default is 'Regular'. | "Regular" "Spot" |
| spotMaxPrice | Possible values are any decimal value greater than zero or -1 which indicates the willingness to pay any on-demand price. For more details on spot pricing, see spot VMs pricing | int |
| tags | The tags to be persisted on the agent pool virtual machine scale set. | object |
| type | The type of Agent Pool. | "AvailabilitySet" "VirtualMachineScaleSets" |
| upgradeSettings | Settings for upgrading the agentpool | AgentPoolUpgradeSettings |
| vmSize | VM size availability varies by region. If a node contains insufficient compute resources (memory, cpu, etc) pods might fail to run correctly. For more details on restricted VM sizes, see: /azure/aks/quotas-skus-regions | string |
| vnetSubnetID | If this is not specified, a VNET and subnet will be generated and used. If no podSubnetID is specified, this applies to nodes and pods, otherwise it applies to just nodes. This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName} | string |
| workloadRuntime | Determines the type of workload a node can run. | "OCIContainer" "WasmWasi" |
CreationData
| Name | Description | Value |
|---|---|---|
| sourceResourceId | This is the ARM ID of the source object to be used to create the target object. | string |
KubeletConfig
| Name | Description | Value |
|---|---|---|
| allowedUnsafeSysctls | Allowed list of unsafe sysctls or unsafe sysctl patterns (ending in *). |
string[] |
| containerLogMaxFiles | The maximum number of container log files that can be present for a container. The number must be ≥ 2. | int Constraints: Min value = 2 |
| containerLogMaxSizeMB | The maximum size (e.g. 10Mi) of container log file before it is rotated. | int |
| cpuCfsQuota | The default is true. | bool |
| cpuCfsQuotaPeriod | The default is '100ms.' Valid values are a sequence of decimal numbers with an optional fraction and a unit suffix. For example: '300ms', '2h45m'. Supported units are 'ns', 'us', 'ms', 's', 'm', and 'h'. | string |
| cpuManagerPolicy | The default is 'none'. See Kubernetes CPU management policies for more information. Allowed values are 'none' and 'static'. | string |
| failSwapOn | If set to true it will make the Kubelet fail to start if swap is enabled on the node. | bool |
| imageGcHighThreshold | To disable image garbage collection, set to 100. The default is 85% | int |
| imageGcLowThreshold | This cannot be set higher than imageGcHighThreshold. The default is 80% | int |
| podMaxPids | The maximum number of processes per pod. | int |
| topologyManagerPolicy | For more information see Kubernetes Topology Manager. The default is 'none'. Allowed values are 'none', 'best-effort', 'restricted', and 'single-numa-node'. | string |
LinuxOSConfig
| Name | Description | Value |
|---|---|---|
| swapFileSizeMB | The size in MB of a swap file that will be created on each node. | int |
| sysctls | Sysctl settings for Linux agent nodes. | SysctlConfig |
| transparentHugePageDefrag | Valid values are 'always', 'defer', 'defer+madvise', 'madvise' and 'never'. The default is 'madvise'. For more information see Transparent Hugepages. | string |
| transparentHugePageEnabled | Valid values are 'always', 'madvise', and 'never'. The default is 'always'. For more information see Transparent Hugepages. | string |
SysctlConfig
| Name | Description | Value |
|---|---|---|
| fsAioMaxNr | Sysctl setting fs.aio-max-nr. | int |
| fsFileMax | Sysctl setting fs.file-max. | int |
| fsInotifyMaxUserWatches | Sysctl setting fs.inotify.max_user_watches. | int |
| fsNrOpen | Sysctl setting fs.nr_open. | int |
| kernelThreadsMax | Sysctl setting kernel.threads-max. | int |
| netCoreNetdevMaxBacklog | Sysctl setting net.core.netdev_max_backlog. | int |
| netCoreOptmemMax | Sysctl setting net.core.optmem_max. | int |
| netCoreRmemDefault | Sysctl setting net.core.rmem_default. | int |
| netCoreRmemMax | Sysctl setting net.core.rmem_max. | int |
| netCoreSomaxconn | Sysctl setting net.core.somaxconn. | int |
| netCoreWmemDefault | Sysctl setting net.core.wmem_default. | int |
| netCoreWmemMax | Sysctl setting net.core.wmem_max. | int |
| netIpv4IpLocalPortRange | Sysctl setting net.ipv4.ip_local_port_range. | string |
| netIpv4NeighDefaultGcThresh1 | Sysctl setting net.ipv4.neigh.default.gc_thresh1. | int |
| netIpv4NeighDefaultGcThresh2 | Sysctl setting net.ipv4.neigh.default.gc_thresh2. | int |
| netIpv4NeighDefaultGcThresh3 | Sysctl setting net.ipv4.neigh.default.gc_thresh3. | int |
| netIpv4TcpFinTimeout | Sysctl setting net.ipv4.tcp_fin_timeout. | int |
| netIpv4TcpkeepaliveIntvl | Sysctl setting net.ipv4.tcp_keepalive_intvl. | int |
| netIpv4TcpKeepaliveProbes | Sysctl setting net.ipv4.tcp_keepalive_probes. | int |
| netIpv4TcpKeepaliveTime | Sysctl setting net.ipv4.tcp_keepalive_time. | int |
| netIpv4TcpMaxSynBacklog | Sysctl setting net.ipv4.tcp_max_syn_backlog. | int |
| netIpv4TcpMaxTwBuckets | Sysctl setting net.ipv4.tcp_max_tw_buckets. | int |
| netIpv4TcpTwReuse | Sysctl setting net.ipv4.tcp_tw_reuse. | bool |
| netNetfilterNfConntrackBuckets | Sysctl setting net.netfilter.nf_conntrack_buckets. | int |
| netNetfilterNfConntrackMax | Sysctl setting net.netfilter.nf_conntrack_max. | int |
| vmMaxMapCount | Sysctl setting vm.max_map_count. | int |
| vmSwappiness | Sysctl setting vm.swappiness. | int |
| vmVfsCachePressure | Sysctl setting vm.vfs_cache_pressure. | int |
ManagedClusterAgentPoolProfilePropertiesNodeLabels
| Name | Description | Value |
|---|---|---|
| {customized property} | string |
PowerState
| Name | Description | Value |
|---|---|---|
| code | Tells whether the cluster is Running or Stopped | "Running" "Stopped" |
AgentPoolUpgradeSettings
| Name | Description | Value |
|---|---|---|
| maxSurge | This can either be set to an integer (e.g. '5') or a percentage (e.g. '50%'). If a percentage is specified, it is the percentage of the total agent pool size at the time of the upgrade. For percentages, fractional nodes are rounded up. If not specified, the default is 1. For more information, including best practices, see: /azure/aks/upgrade-cluster#customize-node-surge-upgrade | string |