Ubuntu 18.04 VMs unable to boot

Question

One of my virtual machines is not able to boot up. After doing some checking found some error messages under Boot Diagnostics -> Serial Log.

[   12.792845] cloud-init[835]: Cloud-init v. 22.2-0ubuntu1~18.04.3 running 'init-local' at Wed, 10 May 2023 15:16:51 +0000. Up 12.06 seconds.
[   12.805351] cloud-init[835]: 2023-05-10 15:16:51,955 - util.py[WARNING]: failed stage init-local
[   12.824038] cloud-init[835]: failed run of stage init-local
[   12.826476] cloud-init[835]: ------------------------------------------------------------
[   12.829934] cloud-init[835]: Traceback (most recent call last):
[   12.832423] cloud-init[835]:   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 740, in status_wrapper
[   12.836791] cloud-init[835]:     ret = functor(name, args)
[   12.839048] cloud-init[835]:   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 410, in main_init
[   12.843124] cloud-init[835]:     init.apply_network_config(bring_up=bring_up_interfaces)
[   12.846489] cloud-init[835]:   File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 938, in apply_network_config
[   12.851123] cloud-init[835]:     netcfg, bring_up=bring_up
[   12.853471] cloud-init[835]:   File "/usr/lib/python3/dist-packages/cloudinit/distros/__init__.py", line 233, in apply_network_config
[   12.858290] cloud-init[835]:     self._write_network_state(network_state)
[   12.860958] cloud-init[835]:   File "/usr/lib/python3/dist-packages/cloudinit/distros/debian.py", line 142, in _write_network_state
[   12.865784] cloud-init[835]:     return super()._write_network_state(network_state)
[   12.869004] cloud-init[835]:   File "/usr/lib/python3/dist-packages/cloudinit/distros/__init__.py", line 129, in _write_network_state
[   12.876466] cloud-init[835]:     renderer.render_network_state(network_state)
[   12.879232] cloud-init[835]:   File "/usr/lib/python3/dist-packages/cloudinit/net/netplan.py", line 259, in render_network_state
[   12.883726] cloud-init[835]:     self._netplan_generate(run=self._postcmds)
[   12.886847] cloud-init[835]:   File "/usr/lib/python3/dist-packages/cloudinit/net/netplan.py", line 266, in _netplan_generate
[   12.891030] cloud-init[835]:     subp.subp(self.NETPLAN_GENERATE, capture=True)
[   12.893836] cloud-init[835]:   File "/usr/lib/python3/dist-packages/cloudinit/subp.py", line 336, in subp
[   12.897484] cloud-init[835]:     stdout=out, stderr=err, exit_code=rc, cmd=args
[   12.900532] cloud-init[835]: cloudinit.subp.ProcessExecutionError: Unexpected error while running command.
[   12.907679] cloud-init[835]: Command: ['netplan', 'generate']
[   12.909909] cloud-init[835]: Exit code: 1
[   12.911571] cloud-init[835]: Reason: -
[   12.913337] cloud-init[835]: Stdout:
[   12.914932] cloud-init[835]: Stderr: Error in network definition /etc/netplan/50-cloud-init.yaml line 8 column 12: unknown key dhcp4-overrides
[   12.920083] cloud-init[835]: ------------------------------------------------------------
[FAILED] Failed to start Initial cloud-init job (pre-networking).
See 'systemctl status cloud-init-local.service' for details.

From the screenshot, it was stuck in the login screen.

I have tried to access the VM from the serial console, but it was asking username and password for login which I didn't have it. I have tried to reset the password but it took forever and I never success in resetting the password.

Answer 1

@Tjaco Walvis

I understand that your Ubuntu 18.04 is not accessible, and you found the above errors in the logs. Thanks for providing the detailed log message.

This issue can happen if you are running an old version of netplan. To resolve this issue you will need to upgrade the version of netplan. Since the VM is not accessible you will need do this via a rescue VM. I am including the steps here:
Stop or deallocate the affected VM.

Create a rescue VM of the same generation, same OS version, Ubuntu 18.04, in same resource group and location using managed disk.

Use the Azure portal to take a snapshot of the affected virtual machine's OS disk.

Create a disk out of the snapshot of the OS disk, and attach it to the rescue VM.

Once the disk has been created, troubleshoot the chroot environment in the rescue VM.

Access your VM as the root user by using the following command:

sudo su -

Find the disk using dmesg (the method you use to discover your new disk may vary). The following example uses dmesg to filter on Small Computer Systems Interface (SCSI) disks:

dmesg | grep SCSI

The command output is similar to the following example. In this example, the /dev/sdc disk is what you want:

[    0.294784] SCSI subsystem initialized

[    0.573458] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)

[    7.110271] sd 2:0:0:0: [sda] Attached SCSI disk

[    8.079653] sd 3:0:1:0: [sdb] Attached SCSI disk

[ 1828.162306] sd 5:0:0:0: [sdc] Attached SCSI disk

Use the following commands to access the chroot environment:

mkdir /rescue
mount /dev/sdc1 /rescue
mount /dev/sdc15 /rescue/boot/efi
mount -t proc /proc /rescue/proc
mount -t sysfs /sys /rescue/sys
mount -o bind /dev /rescue/dev
mount -o bind /dev/pts /rescue/dev/pts
mount -o bind /run /rescue/run
chroot /rescue

Install the netplan package in chroot environment

  apt-get update
  apt install netplan.io

Use the following commands to exit the chroot environment:

exit
umount /rescue/proc/
umount /rescue/sys/
umount /rescue/dev/pts
umount /rescue/dev/
umount /rescue/run
cd /
umount /rescue/boot/efi
umount /rescue

Note

If you receive the error message "unable to unmount /rescue," add the -l option to the umount command, for example, umount -l /rescue.

Detach the disk from the rescue VM and perform a disk swap with the original VM.

Start the original VM and check its connectivity.

Hope this helps! Let me know if you have any follow-up questions or are still facing issues.

---

If you still have questions, please let us know in the "comments" and we would be happy to help you. Comment is the fastest way of notifying the experts.

If the answer has been helpful, we appreciate hearing from you and would love to help others who may have the same question. Accepting answers helps increase visibility of this question for other members of the Microsoft Q&A community.

Thank you for helping to improve Microsoft Q&A!