@Craig Jones , Thank you for your question.
To address the first concern over
spec.template.spec.containers.resources.limits.cpu set to 3 in the
coredns deployment manifest, if the node where a Pod is running has enough of a resource available, it's possible (and allowed) for a container to use more resource than its request for that resource specifies. However, a container is not allowed to use more than its resource limit. Thus at any point, each coredns container is not necessarily allocated 3 vcpu. Instead you can use
kubectl top pod -n kube-system <coredns_pod_name> --containers to get the CPU being utilized by the coredns container at any instant. The limit is a threshold, which if surpassed, the pod will be evicted.
If you do a
kubectl describe deployment -n kube-system coredns, you will find a very interesting label
Now, addons with label
addonmanager.kubernetes.io/mode=Reconcile will be periodically reconciled. Direct manipulation to these addons through apiserver is discouraged because addon-manager will bring them back to the original state. In particular:
- Addon will be re-created if it is deleted.
- Addon will be reconfigured to the state given by the supplied fields in the template file periodically.
- Addon will be deleted when its manifest file is deleted from the
$ADDON_PATH by default is set to
/etc/kubernetes/addons/ on the control plane node(s).
For more information please check this document.
Since AKS is a managed Kubernetes Service you will not be able to access
$ADDON_PATH. We strongly recommend against forcing changes to
kube-system resources as these are critical for the proper functioning of the cluster.
Hope this helps.
Please "Accept as Answer" if it helped, so that it can help others in the community looking for help on similar topics.