What's the recommended course of action when machine deployment is broken

Question

The newly started VM has a problem with Temp storage (SSD).

root@azwork:~# ls -al /mnt/
total 24
dr--r--r-- 3 root root 4096 Mar 14 09:02 .
drwxr-xr-x 19 root root 4096 Mar 14 09:10 ..
drwx------ 2 root root 16384 Mar 14 09:02 lost+found
root@azwork:~#

Something fishy going on with that.

root@azwork:~# echo hello > /mnt/23

-bash: /mnt/23: Operation not permitted

root@azwork:~#

I'm also unable to reformat /dev/sdb1,

root@azwork:/# umount mnt

root@azwork:/# mkfs.ext4 /dev/sdb1

mke2fs 1.46.5 (30-Dec-2021)

/dev/sdb1 is mounted; will not make a filesystem here!

root@azwork:/#

I think the drive is in some kind of read-only mode on the host side. I wasn't able to create a support ticket, sorry.

Answer 1

Hello Antti Rytsola,

Welcome to microsoft Q&A, Thankyou for posting your query here.

Adjust the permissions of the /mnt directory to allow writing and execution.

chmod 755 /mnt

check if the disk is still mounted using

mount | grep /dev/sdb1

mount command to list all mounted filesystems and grep for /dev/sdb1 to see if it appears in the list.

Find out if it’s being used by another process, using lsof or fuser to identify any processes that might be using /dev/sdb1.

lsof /dev/sdb1

fuser -m /dev/sdb1

You can forcly umount using -f and -l tag.

umount -f /dev/sdb1 or umount -f /mnt

umount -l /dev/sdb1 or umount -l /mnt

Try formatting again using

mkfs.ext4 /dev/sdb1

Check for filesystem issues using "fsck"

fsck /dev/sdb1

If all else fails, a reboot can sometimes clear up the issue, especially if the disk is stuck in a busy state due to a system or kernel issue.

Hope this helps you, if an answer has been helpful, please consider accepting the answer to help increase visibility of this question for other members of the Microsoft Q&A community. If not, please let us know what is still needed in the comments so the question can be answered. Thank you for helping to improve Microsoft Q&A!

Answer 2

Seems the mount was hang up in kernel somehow.

While the mount could not be formatted or chmod, the device block driver was not protected.

So either use fdisk to remove and create new partitions ( I did 1G sdb1 and rest in sdb2 ) or use dd to clear first gig of /dev/sdb and reboot.

I still think this belongs to devOps.