VM Provision failing: running out of diskspace during creation of yum cache

Philip Hopkins 1 Reputation point
2021-05-02T22:09:33.563+00:00

My VPN create scripts are suddenly having an issue, without any modification to the script. The Azure provisioning process seems to be filling up the yum cache within /var/cache/yum//.

The error that shows up in the file cloud-init.log is a failure to create the yum cache. The /var partition is 100% full when the provisioning completes, almost all of the space taken up within the yum cache subdirectory. Perhaps the yum make cache is trying to reserve space and wants more than the 8Gb in the partition? No idea why this just started happening however.

From cloud-init.log

ProcessExecutionError: Unexpected error while running command.
Command: ['yum', '-t', '-y', 'makecache']
Exit code: 1
Reason: -
Stdout: -
Stderr: -

In the cloud-init-output.log, there is an immediate failure when trying to install my additional packages:

cloud-init v. 19.4 running 'modules:config' at Sun, 02 May 2021 21:29:53 +0000. Up 302.65 seconds.
Loaded plugins: langpacks, product-id, search-disabled-repos
Error: Error writing to file /var/cache/yum/x86_64/7Server/rhui-rhel-server-rhui-rhscl-7-rpms/gen/filelists.xml: [Errno 28] No space left on device
2021-05-02 21:34:22,956 - util.py[WARNING]: Package update failed

This exact same VM provisioning worked fine for a few months, and now suddenly just fails. I have not change the instance type (Standard_E8s_v3), and I've tried a RHEL 9 and RHEL 8 image.

I can run 'yum clean all' and it frees up all the space, however when I try the provisioning command 'yum -y -t makecache', the directory fills up again and the command fails with:

Error: Error writing to file /var/cache/yum/x86_64/7Server/rhui-rhel-server-rhui-rhscl-7-rpms/gen/filelists.xml: [Errno 28] No space left on device

The OS Disk details are:

93202-image.png

Thanks,
Phil

Azure Virtual Machines
Azure Virtual Machines
An Azure service that is used to provision Windows and Linux virtual machines.
9,014 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.