Custom node configuration parameters for Azure Kubernetes Service (AKS)

This article lists supported custom node configuration parameters for Azure Kubernetes Service (AKS) node pools. To learn how to create and apply configuration files, see Customize the node configuration for AKS node pools.

Kubelet custom configuration parameters

Important

When enabling unsafe sysctls, you assume responsibility for node stability and workload behavior. Unsafe sysctls can potentially cause node instability or security vulnerabilities if misconfigured. Ensure you understand the implications of enabling specific unsafe sysctls and monitor your cluster's health closely after making changes.

Note

The kubeReserved, hardEvictionThreshold, softEvictionThreshold, softEvictionGracePeriod, and evictionMaxPodGracePeriodInSeconds settings are part of the AKS Node Hardening preview and apply only to Linux node pools. Register the CustomNodeConfigPreview feature flag before you use these settings. To enable Node Hardening, see Node Hardening (preview) in AKS.

For each eviction signal, set both the softEvictionThreshold and matching softEvictionGracePeriod values. You can omit both values to use the default or set both values to empty strings to opt out of soft eviction for that signal. If you specify a soft eviction threshold, it must be greater than the effective hard eviction threshold for the same signal.

Linux kubelet custom configuration

Parameter	Allowed values/interval	Default	Description
cpuManagerPolicy	none, static	none	The static policy allows containers in guaranteed pods with integer CPU requests access to exclusive CPUs on the node.
cpuCfsQuota	true, false	true	Enable/disable CPU CFS quota enforcement for containers that specify CPU limits.
cpuCfsQuotaPeriod	Interval in milliseconds (ms)	100ms	Sets CPU CFS quota period value.
imageGcHighThreshold	0-100	85	The percent of disk usage after which image garbage collection is always run. Minimum disk usage that triggers garbage collection. To disable image garbage collection, set to 100.
imageGcLowThreshold	0-100, no higher than imageGcHighThreshold	80	The percent of disk usage before which image garbage collection is never run. Minimum disk usage that can trigger garbage collection.
topologyManagerPolicy	none, best-effort, restricted, single-numa-node	none	Optimize NUMA node alignment. For more information, see Control Topology Management Policies on a node.
allowedUnsafeSysctls	kernel.shm*, kernel.msg*, kernel.sem, fs.mqueue.*, net.*	None	Allowed list of unsafe sysctls or unsafe sysctl patterns.
containerLogMaxSizeMB	Size in megabytes (MB)	50	The maximum size (for example, 10 MB) of a container log file before it gets rotated.
containerLogMaxFiles	2 or greater	5	The maximum number of container log files to retain for a container.
podMaxPids	-1 to kernel PID limit	-1 (infinite)	The maximum number of process IDs that can run in a pod.
seccompDefault	Unconfined, RuntimeDefault	Unconfined	Sets the default seccomp profile for all workloads. RuntimeDefault uses containerd's default seccomp profile, restricting certain system calls to enhance security. Restricted syscalls fail. Unconfined places no restrictions on syscalls, allowing all system calls and reducing security. For more information, see the containerd default seccomp profile. This parameter is in preview. Register the "KubeletDefaultSeccompProfilePreview" feature flag using the az feature register command with --namespace "Microsoft.ContainerService".
kubeReserved.cpuMillicores	140 to the node pool CPU capacity in millicores	AKS-computed based on VM size	Reserves CPU for Kubernetes system daemons.
kubeReserved.memoryMB	750 to the node pool memory capacity in MiB	AKS-computed from node memory and maximum pods per node: min(20 × maxPods + 50, 25% of node memory) MiB. With Node Hardening: min(35 × maxPods + max(250, 2% of node memory), 25% of node memory) MiB	Reserves memory for Kubernetes system daemons.
hardEvictionThreshold.memoryAvailable	<number>Ki, <number>Mi, or <number>Gi with a minimum of 100Mi; or 2% to 100%	100Mi. With Node Hardening: 250Mi for nodes with 8 GiB of memory or less, 375Mi for nodes with more than 8 GiB and less than 32 GiB, and 512Mi for nodes with 32 GiB or more	Sets the hard eviction threshold for available memory.
hardEvictionThreshold.nodeFsAvailable	<number>Ki, <number>Mi, <number>Gi, or 10% to 100%	10%	Sets the hard eviction threshold for available node filesystem space.
hardEvictionThreshold.nodeFsInodesFree	<number> or 5% to 100%	5%	Sets the hard eviction threshold for free node filesystem inodes.
softEvictionThreshold.memoryAvailable	Empty string; <number>Ki, <number>Mi, or <number>Gi with a minimum of 100Mi; or 2% to 100%	None. With Node Hardening: 500Mi for nodes with 8 GiB of memory or less, 750Mi for nodes with more than 8 GiB and less than 32 GiB, and 1Gi for nodes with 32 GiB or more	Sets the soft eviction threshold for available memory. The value must be greater than the effective hardEvictionThreshold.memoryAvailable value.
softEvictionThreshold.nodeFsAvailable	Empty string; <number>Ki, <number>Mi, <number>Gi, or 10% to 100%	12% when Node Hardening is enabled; None otherwise	Sets the soft eviction threshold for available node filesystem space. The value must be greater than the effective hardEvictionThreshold.nodeFsAvailable value.
softEvictionThreshold.nodeFsInodesFree	Empty string; <number> or 5% to 100%	7% when Node Hardening is enabled; None otherwise	Sets the soft eviction threshold for free node filesystem inodes. The value must be greater than the effective hardEvictionThreshold.nodeFsInodesFree value.
softEvictionGracePeriod.memoryAvailable	Empty string or Go-style duration of at least 30s using ns, us, ms, s, m, or h	30s when Node Hardening is enabled; None otherwise	Sets how long the memoryAvailable soft eviction threshold must be crossed before kubelet evicts pods.
softEvictionGracePeriod.nodeFsAvailable	Empty string or Go-style duration of at least 30s using ns, us, ms, s, m, or h	2m when Node Hardening is enabled; None otherwise	Sets how long the nodeFsAvailable soft eviction threshold must be crossed before kubelet evicts pods.
softEvictionGracePeriod.nodeFsInodesFree	Empty string or Go-style duration of at least 30s using ns, us, ms, s, m, or h	2m when Node Hardening is enabled; None otherwise	Sets how long the nodeFsInodesFree soft eviction threshold must be crossed before kubelet evicts pods.
evictionMaxPodGracePeriodInSeconds	0 or greater	60 when Node Hardening is enabled; 0 otherwise	Sets the maximum number of seconds that kubelet allows pods to terminate during a soft eviction.

Windows kubelet custom configuration

Parameter	Allowed values/interval	Default	Description
imageGcHighThreshold	0-100	85	The percent of disk usage after which image garbage collection is always run. Minimum disk usage that triggers garbage collection. To disable image garbage collection, set to 100.
imageGcLowThreshold	0-100, no higher than imageGcHighThreshold	80	The percent of disk usage before which image garbage collection is never run. Minimum disk usage that can trigger garbage collection.
containerLogMaxSizeMB	Size in megabytes (MB)	10	The maximum size (for example, 10 MB) of a container log file before it gets rotated.
containerLogMaxFiles	2 or greater	5	The maximum number of container log files to retain for a container.

Linux custom OS configuration parameters

Important

To simplify search and readability, the OS settings are displayed in this article by their name, but they should be added to the configuration JSON file or AKS API using the camelCase capitalization convention.

For example, if you modify the vm.max_map_count setting, you should reformat it to vmMaxMapCount in the configuration JSON file.

Linux file handle limits

When serving high amounts of traffic, that traffic commonly comes from a large number of local files. You can adjust the following kernel settings and built-in limits to allow you to handle more, at the cost of some system memory.

The following table lists the file handle limits that you can customize per node pool:

Setting	Allowed values/interval	Ubuntu 22.04 default	Ubuntu 24.04 default	Azure Linux 3.0 default	Description
fs.file-max	8192 - 9223372036854775807	9223372036854775807	9223372036854775807	9223372036854775807	Maximum number of file-handles that the Linux kernel allocates. This value is set to the maximum possible value (2^63-1) to prevent file descriptor exhaustion and ensure unlimited system-wide file handles for containerized workloads.
fs.inotify.max_user_watches	781250 - 2097152	1048576	1048576	1048576	Maximum number of file watches allowed by the system. Each watch is roughly 90 bytes on a 32-bit kernel, and roughly 160 bytes on a 64-bit kernel.
fs.aio-max-nr	65536 - 6553500	65536	65536	65536	The aio-nr shows the current system-wide number of asynchronous I/O requests. aio-max-nr allows you to change the maximum value aio-nr can grow to.
fs.nr_open	8192 - 20000500	1048576	1048576	1073741816	The maximum number of file-handles a process can allocate.

Note

The fs.file-max parameter is set to 9223372036854775807 (the maximum value for a signed 64-bit integer) across Ubuntu and Azure Linux based on upstream defaults. This configuration:

• Prevents denial-of-service attacks based on system-wide file descriptor exhaustion.
• Ensures container workloads are never bottlenecked by system-wide file handle limits.
• Maintains security through per-process limits (fs.nr_open and ulimit) which still apply to individual processes.
• Optimizes for container platforms where many containers might run simultaneously, each potentially opening many files and network connections.

Linux socket and network tuning

For agent nodes, which are expected to handle large numbers of concurrent sessions, you can use following TCP and network options and adjust them per node pool:

Setting	Allowed values/interval	Ubuntu 22.04 default	Ubuntu 24.04 default	Azure Linux 3.0 default	Description
net.core.somaxconn	4096 - 3240000	16384	16384	16384	Maximum number of connection requests that can be queued for any given listening socket. An upper limit for the value of the backlog parameter passed to the listen(2) function. If the backlog argument is greater than the somaxconn, then it's silently truncated to this limit.
net.core.netdev_max_backlog	1000 - 3240000	1000	1000	1000	Maximum number of packets, queued on the INPUT side, when the interface receives packets faster than kernel can process them.
net.core.rmem_max	212992 - 134217728	1048576	1048576	212992	The maximum receive socket buffer size in bytes.
net.core.wmem_max	212992 - 134217728	212992	212992	212992	The maximum send socket buffer size in bytes.
net.core.optmem_max	20480 - 4194304	20480	131072	20480	Maximum ancillary buffer size (option memory buffer) allowed per socket. Socket option memory is used in a few cases to store extra structures relating to usage of the socket.
net.ipv4.tcp_max_syn_backlog	128 - 3240000	16384	16384	16384	The maximum number of queued connection requests that haven't received an acknowledgment from the connecting client. If this number is exceeded, the kernel begins dropping requests.
net.ipv4.tcp_max_tw_buckets	8000 - 1440000	262144	262144	131072	Maximum number of TIME-WAIT sockets held by the system simultaneously. If this number is exceeded, time-wait socket is immediately destroyed and warning is printed.
net.ipv4.tcp_fin_timeout	5 - 120	60	60	60	The length of time an orphaned (no longer referenced by any application) connection remains in the FIN_WAIT_2 state before it's aborted at the local end.
net.ipv4.tcp_keepalive_time	30 - 432000	7200	7200	7200	How often TCP sends out keepalive messages when keepalive is enabled.
net.ipv4.tcp_keepalive_probes	1 - 15	9	9	9	How many keepalive probes TCP sends out, until it decides that the connection is broken.
net.ipv4.tcp_keepalive_intvl	10 - 90	75	75	75	How frequently the probes are sent out. Multiplied by tcp_keepalive_probes it makes up the time to kill a connection that isn't responding, after probes started.
net.ipv4.tcp_tw_reuse		2	2	2	Allows reuse of TIME-WAIT sockets for new connections when it's safe from a protocol viewpoint.
net.ipv4.ip_local_port_range	First: 1024 - 60999 and Last: 32768 - 65535]	First: 32768 and Last: 60999	First: 32768 and Last: 60999	First: 32768 and Last: 60999	The local port range that is used by TCP and UDP traffic to choose the local port. The range is composed of two numbers: the first local port allowed for TCP and UDP traffic on the agent node, and the last local port number.
net.ipv4.neigh.default.gc_thresh1	128 - 80000	4096	4096	4096	Minimum number of entries that can be in the ARP cache. Garbage collection isn't triggered if the number of entries is below this setting.
net.ipv4.neigh.default.gc_thresh2	512 - 90000	8192	8192	8192	Soft maximum number of entries that can be in the ARP cache. This setting is arguably the most important, as ARP garbage collection triggers about 5 seconds after reaching this soft maximum.
net.ipv4.neigh.default.gc_thresh3	1024 - 100000	16384	16384	16384	Hard maximum number of entries in the ARP cache.
net.netfilter.nf_conntrack_max	131072 - 2097152	Dynamically calculated	Dynamically calculated	Dynamically calculated	nf_conntrack is a module that tracks connection entries for NAT within Linux. The nf_conntrack module uses a hash table to record the established connection record of the TCP protocol. nf_conntrack_max is the maximum number of nodes in the hash table, that is, the maximum number of connections supported by the nf_conntrack module or the size of connection tracking table. Default value is dynamically calculated based on system memory using the formula: RAM_in_bytes / 16384 (or RAM_in_MB * 64). For example, a virtual machine with 8 GB RAM has a default of approximately 524,288 connections. Actual values vary based on the virtual machine size and available memory.
net.netfilter.nf_conntrack_buckets	65536 - 524288	Dynamically calculated	Dynamically calculated	Dynamically calculated	nf_conntrack is a module that tracks connection entries for NAT within Linux. The nf_conntrack module uses a hash table to record the established connection record of the TCP protocol. nf_conntrack_buckets is the size of hash table. Default value is dynamically calculated based on system memory using the formula: RAM_in_bytes / 16384, with a minimum of 1,024 buckets and a maximum of 262,144 buckets. The default nf_conntrack_max is typically set to nf_conntrack_buckets * 4. Actual values vary based on the virtual machine size and available memory.

Linux worker limits

Like file descriptor limits, the number of workers or threads that a process can create are limited by both a kernel setting and user limits. The user limit on AKS is unlimited. The following table lists the kernel setting that you can customize per node pool:

Setting	Ubuntu 22.04 default	Ubuntu 24.04 default	Azure Linux 3.0 default	Description
kernel.threads-max	Dynamically calculated	Dynamically calculated	Dynamically calculated	Processes can spin up worker threads. The maximum number of all threads that can be created is set with the kernel setting kernel.threads-max. Default value is dynamically calculated based on system memory using the formula: total_ram_pages / 4 (where each page is typically 4 KB). Actual values vary based on the virtual machine size and available memory.

Linux virtual memory

The following table lists the kernel settings that you can customize per node pool to tune the operation of the virtual memory (VM) subsystem of the Linux kernel and the writeout of dirty data to disk:

Setting	Allowed values/interval	Ubuntu 22.04 default	Ubuntu 24.04 default	Azure Linux 3.0 default	Description
vm.max_map_count		65530	1048576	1048576	This file contains the maximum number of memory map areas a process can have. Memory map areas are used as a side-effect of calling malloc, directly by mmap, mprotect, and madvise, and also when loading shared libraries.
vm.vfs_cache_pressure	1 - 100	100	100	100	This percentage value controls the tendency of the kernel to reclaim the memory, which is used for caching of directory and inode objects.
vm.swappiness	0 - 100	60	60	60	This control is used to define how aggressively the kernel swaps memory pages. Higher values increase aggressiveness, lower values decrease the amount of swap. A value of 0 instructs the kernel not to initiate swap until the amount of free and file-backed pages is less than the high water mark in a zone.
swapFileSizeMB	1 MB - Size of the temporary disk (/dev/sdb)	None	None	None	SwapFileSizeMB specifies the size in MB of a swap file to create on the agent nodes from this node pool.
transparentHugePageEnabled	always, madvise, never	always	always	madvise	Transparent Hugepages is a Linux kernel feature intended to improve performance by making more efficient use of your processor's memory-mapping hardware. When enabled the kernel attempts to allocate hugepages whenever possible and any Linux process receives 2-MB pages if the mmap region is 2 MB naturally aligned. In certain cases when hugepages are enabled system wide, applications might end up allocating more memory resources. An application might mmap a large region but only touch 1 byte of it, in that case a 2-MB page might be allocated instead of a 4k page for no good reason. This scenario is why it's possible to disable hugepages system-wide or to only have them inside MADV_HUGEPAGE madvise regions.
transparentHugePageDefrag	always, defer, defer+madvise, madvise, never	madvise	madvise	madvise	This value controls whether the kernel should make aggressive use of memory compaction to make more hugepages available.