Customize worker nodes with a DaemonSet
To meet application requirements, you may need to modify operating system settings, enable a Linux kernel module or install a host-level application package. Use a DaemonSet
with host privileges to customize worker nodes.
The example DaemonSet
sets registry.contoso.com
to bypass the Cloud Services Network proxy for image pulls, installs the SCTP kernel module and sets fs.inotify.max_user_instances
to 4096
. Finally, the script applies a label to the Kubernetes Node to ensure the DaemonSet only runs once.
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: customized
namespace: kube-system
spec:
selector:
matchLabels:
name: customized
template:
metadata:
labels:
name: customized
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: customized
operator: NotIn
values:
- "1"
tolerations:
- operator: Exists
effect: NoSchedule
containers:
- name: customized
image: mcr.microsoft.com/cbl-mariner/base/core:1.0
command:
- nsenter
- --target
- "1"
- --mount
- --uts
- --ipc
- --net
- --pid
- --
- bash
- -exc
- |
sed -i '/registrycontoso.com/!s/NO_PROXY=/®istry.contoso.com,/' /etc/systemd/system/containerd.service.d/http-proxy.conf
systemctl daemon-reload
systemctl restart containerd
modprobe sctp
sed -i 's/^fs.inotify.max_user_instances.*/fs.inotify.max_user_instances = 4096/' /etc/sysctl.d/90-system-max-limits.conf
kubectl --kubeconfig=/etc/kubernetes/kubelet.conf label node ${HOSTNAME,,} customized=1
sleep infinity
resources:
limits:
memory: 200Mi
requests:
cpu: 100m
memory: 16Mi
securityContext:
privileged: true
hostNetwork: true
hostPID: true
hostIPC: true
terminationGracePeriodSeconds: 0
And apply the Daemonset
:
kubectl apply -f /path/to/daemonset.yaml