Ubuntu VM stuck in boot

Question

Hi All, we are a small non-profit. I am looking after our apache2-based website that I had installed on a Ubuntu 18.04 VM.
A few days ago, the machine prompted me to migrate/upgrade it to Ubuntu 20.04 and I unfortunately relented and clicked yes. The download/migration seemed to complete without error and asked to reboot. After I restarted the VM, it seems to be stuck in boot. Azure first showed me it's stuck with this error symbol 'grub_file_filters' not found.

I could not connect to the VM and restarted it on Azure portal. It's still stuck in boot somewhere; the only information from azure is this "apology" and I am lost about what to do:

"We identified that a live migration was performed on the VM [VMName] by Azure. After the live migration, the network stack for the VM failed to initialize as expected, at approximately [FailureTimestamp]. This prevented connectivity to the VM from outside the virtual network topology. A fix for this bug was already being deployed to production and is expected to complete in ~4 weeks.

Recommended Steps
Redeploying affected VMs (instructions available for Linux and Windows) mitigates the issue.

We sincerely apologize for the inconvenience. We are continuously taking steps to help ensure such incidents do not occur in the future. In this case, this includes a fix for the code bug being tested and applied to all affected Azure regions, and a review of auto-detection and auto-mitigation logic to address this issue before customers experience impact."

Answer 1

Apologies for the inconvenience caused due to this issue.

As the root cause as well as recommendation on next steps is shared by Azure on the portal, please follow the steps recommended to resolve this issue.

Recommended Steps
Redeploying affected VMs (instructions available for Linux and Windows) mitigates the issue.

If you see any difficulties, please let me know and I can further investigate or involve the right team to help.

Documents to refer:
https://learn.microsoft.com/en-us/troubleshoot/azure/virtual-machines/redeploy-to-new-node-linux
https://learn.microsoft.com/en-us/troubleshoot/azure/virtual-machines/redeploy-to-new-node-windows

Thanks.

Answer 2

I tried redeploying AND reapplying but it didn't help.
However, I chanced upon a method to create a repair VM and then reinstall the grub on the affected OS disk via chrooting.

That pretty much fixed the problem. Thanks

Answer 3

Hello. We had an exact same issue with our Ubuntu 18.04 VM after update to 20.04 (error: symbol grub_file_filters not found. Entering rescue mode...).

Is there any official plan on how to avoid it on updating? Like what we need to check or execute after or prior do-release-upgrade to prevent boot stuck after reboot?

Redeploy to a new node was not helpful for us, only grub recovery using chroot from another VM - but that is not the best solution.

The only case that works without issues so far is to execute "sudo grub-install /dev/sdX" and "sudo update-grub" right after do-release-upgrade before reboot. But I am not sure if it will work all the time.