The usage is normal from the output of ( df -h ), but it seems in the screenshot your selection is to show the Sum not the Average, can you select the avg?
Azure AKS - Disk bytes used suddenly take GB/TB
Dear Community,
Since the 21 july 2023 at 8pm, for a reason I ignore (not any change on the AKS infrastructure), the disk used bytes switch from some bytes to 800GB for the system pool and more than 2.5TB for the user pool.
How can I explain this increase since the OS disk size is only 128GB ?
I checked a df -h on the nodes and I can find a lot of overlay files taking lots of GB. Is that normal ?
root@aks-agentpool-95240818-vmss000005:/# df -H
Filesystem Size Used Avail Use% Mounted on
overlay 134G 24G 110G 18% /
tmpfs 68M 0 68M 0% /dev
/dev/root 134G 24G 110G 18% /host
devtmpfs 2.1G 0 2.1G 0% /host/dev
tmpfs 2.1G 0 2.1G 0% /dev/shm
tmpfs 824M 2.2M 822M 1% /host/run
tmpfs 5.3M 0 5.3M 0% /host/run/lock
shm 68M 0 68M 0% /host/run/containerd/io.containerd.grpc.v1.cri/sandboxes/21088829b6b292dcf36a00e0281abfd74a17a9ba4b1cd73aaec3b2d0036c6e9c/shm
shm 68M 0 68M 0% /host/run/containerd/io.containerd.grpc.v1.cri/sandboxes/ce4103bd0bb0e4dba6f6c29068a9868e9e7cf0083f0bbf59ce7ad526f3e17636/shm
shm 68M 0 68M 0% /host/run/containerd/io.containerd.grpc.v1.cri/sandboxes/2e76c3c92b236f261dbc60427aa687f6f123667578349ea617ce6dcd3eebb365/shm
shm 68M 0 68M 0% /host/run/containerd/io.containerd.grpc.v1.cri/sandboxes/6196396872ab86bd09836e83ba36e8cf1bb0ff1d0a60bec4e868909dd349709d/shm
shm 68M 0 68M 0% /host/run/containerd/io.containerd.grpc.v1.cri/sandboxes/cc5e5cd2f11691d894cd9ef79c3bb60ad6941c5c962bba7767f4bf74255bd542/shm
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/ce4103bd0bb0e4dba6f6c29068a9868e9e7cf0083f0bbf59ce7ad526f3e17636/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/2e76c3c92b236f261dbc60427aa687f6f123667578349ea617ce6dcd3eebb365/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/cc5e5cd2f11691d894cd9ef79c3bb60ad6941c5c962bba7767f4bf74255bd542/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/6196396872ab86bd09836e83ba36e8cf1bb0ff1d0a60bec4e868909dd349709d/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/21088829b6b292dcf36a00e0281abfd74a17a9ba4b1cd73aaec3b2d0036c6e9c/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/7d51b9093ed4d8665cf759ebead7e4d17c10bc235c77393db3b4354b11f22a92/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/a98e0a8fa73acd9e440cf2869107ea2f5a66e1d461d3e1ba0a24c707cbf445ac/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/6c0060027da6483dab55529323bf50e19a5c65da65a974567f70c564bee0b60e/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/5e0e35f60231c224fe3032d1146304af130f81c54752e121782e32c8b87554a0/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/2d9806d118955a82ae98fbe7c1dcfbd27a803b52d48b0dacc7eb9cc1de1a91e4/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/f8207c530e10f5960fdd02663014c467bb7b3b4d6fbc0adf255b411d375cb1cd/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/3bfbb51d475ff69620b47bc352707aa9714c6cdc3fe3b82d4a7aaf0f58df2752/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/4257978d9bac2be7111dc467d97ce570b8dd356f8a55d9cd56b21c1ff5662894/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/7cd8921cedd0d1f690510dcb66e1cfa489b5217482413f3779b7ccecc83b7dda/rootfs
overlay 134G 24G 110G 18% /host/run/containerd/io.containerd.runtime.v2.task/k8s.io/445edc955dab77744c4c43039739f803ea660a1421bcffb1b661f4aa2ab9b423/rootfs
Thanks for your help
-
mutaz-msft 2,346 Reputation points Microsoft Employee
2023-07-31T15:24:54.1133333+00:00